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Executive Summary 



This study compared more than 1,000 high school students who participated in service-learning 
programs with those who did not participate in schools matched for similar demographics and 
student achievement profiles. The intention was to estimate the effects of service-learning 
compared to more traditional ways of teaching similar subject areas. The outcomes measured 
ranged from civic knowledge, behaviors, and dispositions to school engagement factors, such as 
attachment to school and enjoyment of coursework, that generally predict academic success. 
Students came from sites in the southeast, north central, and western parts of the United States. 
More than half of the students in the sample were Latino/Hispanic. 

Key Results 

Although service-learning students scored higher than comparison students on several outcomes, 
most of the differences were not statistically significant. Service-learning students were 
significantly more likely to say that they intended to vote and that they enjoyed school. 

There were substantial differences in outcomes among the various service-learning programs in 
the study. The study suggests that service-learning is effective when it is implemented well, but 
it is no more effective than conventional social studies classes when the conditions are not 
optimal. In particular: 

o Student outcomes improved when service-learning programs lasted longer, although 
year-long programs often had slightly less benefit than semester-long programs. 

o Teacher characteristics were related to outcomes. Number of years of teaching 

experience was significantly related to some student outcomes (valuing school, enjoying 
math and science, civic skills, and civic dispositions). Longer experience using service- 
learning was associated with higher civic knowledge, civic dispositions, and efficacy 
scores 

o The type of service project was related to the outcomes. Students who engaged in direct 
service (e.g., tutoring or visiting seniors) were most attached to their communities. 
Students who engaged in indirect service (e.g., fundraising or research) showed the 
highest levels of academic engagement. Students who engaged in political or civic action 
(e.g., circulating a petition or organizing a community forum) scored highest on civic 
knowledge and civic dispositions. 

o The literature has identified a set of best practices or “Essential Elements” of service- 
learning. The study found that some of these elements were related to positive student 
outcomes, but some were not. Results related to this aspect of the study are presented in 
the Exhibit below. 

Teachers who did not use service-learning were almost as likely as those who did to use “active” 
instructional techniques. Thus, based on this study, it does not appear that service-learning 
competes against passive, lecture-style classes. Instead, it competes against mixed forms of 
instruction that include student projects, presentations, debates, and field trips. The study found 
that the use of active teaching techniques was most beneficial for student outcomes, and service- 
learning conferred a small additional benefit over other active pedagogies. 
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Exhibit. Reiationship Between Service-Learning Quaiity Eiements and Student Outcomes 





Elements of Service-Learning Practice 


Significant Positive Relationships 


Significant Negative Relationships 


1. 


Clear educational 2 oals that require the aoDlication of conceuts. 
content, and skills from the academic disciplines and involves 
students in the construction of their own knowledge. 


Civic knowledge, civic dispositions, efficacy, civic 
engagement 


Community attachment 


2. 


Students are encouraged in tasks that challenge and stretch them 
cognitivelv and develonmentallv. 


School attachment, civic skills, efficacy, civic 
engagement 




3. 


Assessment is used as a wav to enhance student learning as well 
as to document and evaluate how well students have met content 
and skills standards. 


School attachment, enjoy reading/language/social 
studies, community attachment 


Civic dispositions 


4. 


Students are engaged in service tasks that have clear goals, meet 
genuine needs in the school or communitv. and have significant 
consequences for themselves and others. 


School attachment, civic knowledge, civic skills, 
civic dispositions, civic engagement 




5. 


Emplovs formative and summative evaluation in a svstematic 
evaluation of the service effort and its outcomes. 




Valuing school, civic dispositions, efficacy 


6. 


Seeks to maximize student voice in selecting, designing, 
implementing, and evaluating the service-learning project. 


Civic knowledge, self-reported civic knowledge 


Subject matter enjoyment 


7. 


Values diversity through its participants, its practice, and its 
outcomes. 


Academic engagement, valuing school, enjoy 
math/science, enjoy reading/language/social studies, 
civic dispositions, civic engagement 


Community attachment 


8. 


Promotes communication and interaction with the communitv 
and encourages partnerships and collaboration. 


School attachment, civic knowledge, self-reported 
civic knowledge, community attachment, civic 
engagement 


Valuing school, enjoy math/science 


9. 


Students are ureuared for all aspects of their service work 
including a clear understanding of task and role, the skills and 
information required by the task, awareness of safety 
precautions, as well as knowledge about and sensitivity to the 
people with whom they will be working. 


Academic engagement, valuing school, enjoy 
math/science, enjoy reading/language/social studies, 
civic dispositions, civic engagement 


Civic knowledge 


10. 


Student reflection takes place before, during and after service, 
uses multiple methods that encourage critical thinking, and is a 
central force in the design and fulfillment of curricular 
objectives. 


School attachment, civic knowledge, self-reported 
civic knowledge, civic skills, civic engagement 


Valuing school, enjoy math/science 


11. 


Multiple methods are designed to acknowledge, celebrate and 
further validate students' service work. 




Enjoyment of subject matters 
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Introduction 



Recent evidence suggests that there is a growing problem of civic disengagement among youth 
in the United States. Young people in high school report having little interest in civic and 
political affairs and little knowledge of or trust in the political system (Levine & Lopez, 2002; 
National Commission on Service-Learning, 2001; Rahm & Transue, 1998; Tomey-Purta, 2002). 
Results from a recent poll indicate that many young people do not feel they can make a 
difference, solve problems in their communities, or have a meaningful impact on politics or 
government (Lake Snell Perry & Associates and The Tarrance Group, Inc, 2002). Policymakers 
and educational leaders alike have noted the woeful lack of interest in civic activities among 
youth and express concern about the future of democracy (for example. Education Commission 
of the States, 2002; National Commission on Service-Learning, 2001). Lack of engagement in 
the political system is particularly pronounced for young women and urban youth (Niemi & 

Junn, 1998; Hart & Atkins, 1992). There are also differences between students based on 
achievement levels. Research has shown that students with a stronger record of academic 
achievement demonstrate greater political knowledge (Niemi & Junn, 1998), and higher rates of 
community participation (Nolin, Chaney, Chapman, & Chandler, 1997) than those with lower 
achievement levels. 

Interestingly, this decline in civic engagement has been paralleled by an increase in volunteerism 
by young people. Studies estimated that over half of young people participate in voluntary 
service (Skinner & Chapman, 1999). As Putnam (2000) optimistically remarked, 

A wide range of evidence . . . suggests that young Americans in the 1990s 
displayed a commitment to volunteerism without parallel among their immediate 
predecessors. This development is the most promising sign of any that I have 
discovered that America might be on the cusp of a new period of civic renewal, 
especially if this youthful volunteerism persists into adulthood and begins to 
expand beyond individual caregiving to broader engagement with social and 
political issues, (p. 13) 

Groups, such as Campus Compact (2000), believed that this is part of the “new student politics” 
and that education may be able to capitalize on the willingness of young people to volunteer as a 
way to engage in active teaching of those civic concepts necessary for the health and well-being 
of American democracy. 

The school-based practice of service-learning offers a promising approach to the challenge of 
civic engagement since it capitalizes both on students’ willingness to volunteer and on the 
options presented in community settings for students to become civically engaged. Service- 
learning is broadly defined as a teaching strategy wherein students learn important curricular 
objectives by providing service that meets authentic community needs. Typically the service- 
learning cycle includes student planning, action, reflection, and celebration. In high quality 
service-learning projects, students have considerable voice in determining activities, and teachers 
facilitate knowledge and skill acquisition. According to Skinner and Chapman (1999), service- 



RMC Research Corporation, Denver, CO 



3 



The Impact of Participation in Service-Learning on 
High Schooi Students’ Civic Engagement 



learning is practiced in half of all public high schools. Private school participation is estimated 
to be even stronger, with nearly 80% implementing service-learning (Pritchard, 2002). 

Several studies have provided evidence for an association between participation in service- 
learning and civic outcomes. A gradually accumulating body of evidence suggests that service- 
learning helps students develop knowledge of community needs, commit to an ethic of service, 
develop more sophisticated understandings of politics and morality, gain a greater sense of civic 
responsibility and feelings of efficacy, and increase their desire to become active contributors to 
society (Billig, 2000; Westheimer & Kahne, 2000; Youniss & Yates, 1997; Youniss, McLellan, 
& Yates, 1997). Studies have provided evidence for positive effects of service-learning on: 

• Civic-related knowledge, including awareness of community needs (Berkas, 1997; Melchior, 
1999; Morgan & Streb, 1999) and knowledge about government (Berkas, 1997; Hamilton & 
Zeldin, 1987; Morgan & Streb, 1999). 

• Civic-related skills, including an understanding of how to design and implement a service 
project (Melchior, 1999). 

• Civic attitudes, including aspects of social and personal responsibility like students’ concern 
for social issues (Metz, McLellan, & Youniss, 2000) and concerns for others’ welfare 
(Scales, Blyth, Berkas, & Kielsmeier, 2000), the belief that they can make a difference in 
their communities (Hamilton & Zeldin, 1987; Melchior, 1999; Morgan & Streb, 1999; Scales 
et ah, 2000), and acceptance of diversity (Melchior, 1999; Morgan & Streb, 1999). 

• Service behavior (Melchior, 1999) and intentions to serve in the future (Berkas, 1997; Metz 
et ah, 2000; Morgan & Streb, 1999). 

• Social capital, including increased connections to schools and other organizations and 
increased social networks (Morgan & Streb, 1999). 

Service-learning experiences may be expected to influence civic identity formation and related 
values and attitudes through the opportunities it provides for students to tackle community 
problems, feel efficacious, and express social responsibility and through benefits generated for 
the community. Similarly, social capital may be advanced through the ways in which service 
projects promote the development of social networks or provide experiences of being mentored 
by an onsite supervisor or other community members. By sharing work responsibilities and 
objectives, feelings of solidarity may be increased with classmates, teachers, and other school 
personnel. 

Given the wide range of types of service-learning programs and the range of quality within each 
type of program, however, programs can be expected to vary substantially in the extent to which 
they are successful in promoting civic identity and engagement. While the potential for service- 
learning to impact civic engagement is considerable, the research shows that unless certain 
practices within service-learning are in place, the impact may not be maximized. In this regard, 
quality components such as presence, amount, frequency, and duration of reflection activities 
matter in the outcomes obtained, along with linkage to standards, direct contact with those being 
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served, and youth voice in planning and implementing the service (Meyer, Billig & Hofschire, 
2004). In addition, the literature suggests that duration, nature, and type of service-learning 
activities could influence outcomes. The individual teacher may also be particularly important in 
facilitating implementation of quality practice. Through his/her curricular decisions and actions, 
the teacher can mediate students’ exposure to particular concepts of citizenship as well as serving 
as a model for how citizenship is enacted. 

This study examined the impact of participation in service-learning on high school students’ 
civic engagement. Using a national sample of classrooms of students that participated in service- 
learning matched with classrooms of students of similar demographic and achievement 
background that do not participate in service-learning, the study investigated the effect of 
service-learning participation on students’ civic knowledge, skills, dispositions, and activities 
and the degree to which many of the variables related to the service-learning experience served 
as moderators of outcomes. 
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Methodology 



The study addressed the following research questions. 

1 . To what extent do students that participate in service-learning show increases over time on 
measures of a variety of aspects of civic engagement and academic and civic knowledge and 
skill acquisition as compared with students in the same or matched schools that participate in 
classes on the same subject matter and do not participate in service-learning? 

2. To what extent do aspects of program quality and other features of the service-learning 
experience moderate the association between participation in service-learning and civic 
outcomes? What teacher characteristics and practices serve to moderate outcomes? 

Sites were selected for participation in this study based on a two step process. First, sites were 
nominated by several service-learning experts, including active researchers and several state 
Learn and Serve directors. Teachers from each of the nominated sites were interviewed by 
telephone to determine: 

• The potential interest in participation; 

• History of service-learning in the school; 

• Type and quality of service-learning implementation; 

• Qualification of the service-learning teachers and descriptions of the service-learning they 
facilitated; 

• Availability of comparison teachers; and 

• Any indicators of impact. 

Nominations from the first pool were analyzed and potential sites for the study were identified. 

A second screening call was then made to determine the ability to commit to the study and to 
engage a comparison teacher for the study that taught the same subject matter, and grade level 
but that did not implement service-learning. The comparison teacher had to teach students with 
approximately the same demographic and achievement profile as the service-learning teacher so 
that the effects of demographics and previous achievement would be minimized. Respondents 
were also asked what would be needed to obtain approval to participate in the study from the 
district and school administration. The final sample was selected on the basis of the reported 
quality of the site, the availability of a comparison site, and the approval of the school and 
district, as well as an effort to gain a broadly representative sample from different regions of the 
country, locales, and socioeconomic, and ethnic backgrounds. As shown in Exhibit 1, the 
sample included: 

• Academy High School 9th-12th-grade television production class, two 9th-12th-grade 
environmental science classes, a 9th-12th-grade art class matched with Dunhar High School 
9th-12th-grade television production class, 9th-12th-grade environmental science class, and 
9th-12th-grade art class. Both schools are located in the Fort Myers, Florida school district. 
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• Anoka High School 12th-grade government and economics class and 1 Ith-grade modern 
world history class matched with Coon Rapids High School 12th-grade economics and 
government class and 1 Ith-grade world history class. Both schools are part of the Anoka- 
Hennepin, Minnesota school district. 

• Homestead Senior High School matched service-learning with non service-learning 
classrooms in 11th- and 12th-grade American government classes; Miami Coral Park 
Senior High School matched six service-learning with six non service-learning classes in 
12th-grade government and economics; the Miami Lakes Educational Center matched four 
service-learning with four non service-learning classes in 11th- and 12th-grade world history 
and government; South Miami High School matched four service-learning and two non 
service-learning 11th- and 12th-grade American government classes; and William G. 

Turner Technical Arts High School matched three service-learning and three non service- 
learning 1 1th- and 12th-grade English classes. All of these schools are located in the Miami- 
Dade County, Florida school district. 

• Quest High School 12th-grade senior project class matched with Humhle High School 
12th-grade government class and 12th-grade economics class. All schools are in the Flumble, 
Texas school district. 

• Tillamook High School service-learning senior class matched with the senior class at Neah- 
Kah-Nie High School. The schools are located in the Tillamook, Oregon, school district and 
the Rockaway Beach, Oregon school district, respectively. 

Data Analysis 

A two (pre-test versus post-test) by two (service-learning versus comparison) mixed multivariate 
analysis of variance (MANOVA) model, with pre-test and post-test as repeated measures was 
initially used to analyze the effects of service-learning on subscale scores and individual survey 
items. When MANOVA tests yielded significant findings, follow-up analysis of covariance 
(ANOVAs) were conducted to explore the results. In order to control for student grade level and 
subject area, hierarchical MANOVAS with repeated measures using nested pairs (service- 
learning versus comparison groups at each study site) were subsequently used to analyze student 
outcomes, and ANOVAs were used to explore results of statistically significant MANOVAs. 
Multivariate t tests were also used to compare student groups when appropriate. 

In order to determine the contribution of particular moderating variables (e.g., engagement in 
service-learning) to the outcomes, two approaches were used. Multivariate regression analyses 
were conducted to determine relationships between more than one continuous predictor variables 
and a dependent variable. When multiple independent, as well as dependent measures were 
involved, a canonical correlation approach was used. Results of statistically significant 
canonical correlational analyses were followed by bivariate correlational analyses to further 
explore the results. 
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Exhibit 1. Sampie Site Characteristics 



Service- 



Number of 
Service- 



Number of 





Learning High 


Comparison 






Grade 


Learning 


Comparison 


Site 


Schooi 


Schooi 




Subject Matter 


Levels 


Classes 


Classes 


Fort Myers, 


Academy 


Dunbar 


• 


TV production 


9-12 


4 


3 


FL 






• 


Environmental 
















science 














• 


Art 








Anoka- 


Anoka 


Coon Rapids 


• 


Government 


11-12 


2 


2 


Hennepin, 








and 








MN 








economics 














• 


World history 








Miami-Dade, 


Homestead 


Homestead 


• 


American 


11-12 


1 


1 


FL 


Senior 


Senior 




government 










Miami Coral 


Miami Coral 


• 


Government 


12 


6 


6 




Park 


Park 




and 
















economics 










Miami Lakes 


Miami Lakes 


• 


World history 


11-12 


4 


4 




Educational 


Educational 




and 










Center 


Center 




government 










South Miami 


South Miami 


• 


American 


11-12 


4 


2 










government 










William G. 


William G. 


• 


English 


11-12 


3 


3 




Turner 


Turner 












Technical 


Technical 












Humble, TX 


Quest 


Humble 


• 


Senior project 


12 


1 


2 










matched with 
















government 

and 
















economics 








Tillamook/ 


Tillamook 


Neah-Kah-Nie 


• 


Senior class 


12 


1 


1 



Rockaway 
Beach, OR 



Methods of Data Coiiection 

The study utilized a mixed-method approach to addressing the research questions, involving the 
collection of both quantitative and qualitative data. 

Student surveys were administered in fall and spring of the 2003-2004 school year. In addition 
to questions about student characteristics, student pre-surveys included measures of civic 
outcomes and attitudes toward school. Civic outcomes addressed by the surveys comprised 
civic knowledge, skills, dispositions, and civic engagement. 

• Civic knowledge was measured through items from the National Assessment of Academic 
Progress (NAEP), about government institutions, leaders and so forth, and the like, as well as 
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a single question about loeal eommunity service organizations. A second knowledge 
measure (“self-reported civic knowledge”) also asked students to rate how well informed 
they were about various aspects of politics (e.g., registering to vote, the difference between 
democracy and socialism, etc.). 

• Civic skills were assessed by asking students to rate their ability to perform particular 
activities required for effective civic participation such as their ability to lead others, conduct 
a campaign to get someone elected, or work to solve a community problem. 

• The measure of community attachment tapped indicators of positive student community 
feeling, such as contributing to, taking pride in, or being viewed as a valued part of the 
neighborhood or local community. 

• The civic dispositions measure assessed civic responsibility; for example, through questions 
about the degree to which students acted to help the needy. 

• Student efficacy was measured by two items that addressed feelings of making a difference 
and having adult responsibilities. 

• Finally, the measure of civic engagement measured political and civic participation such as 
how often students discussed politics, attended rallies, raised funds for a cause, or wrote 
letters to public officials. 

Academic outcomes addressed in the study included academic engagement, valuing school, 
school attachment, enjoyment of math and science, and enjoyment of reading, language arts and 
social studies. 

• The measure of academic engagement asked students about the degree to which they were 
cognitively, affectively and behaviorally engaged in school. 

• The valuing school items were designed to capture the extent to which students felt that 
school work was meaningful and important. 

• The school attachment measure assessed students’ sense of connection to school (e.g., 
belonging, making a contribution, doing things to make the school a better place). 

• Several items asked students about the extent to which they enjoyed specific school subjects. 

Subscales were created for each of the constructs. Item analyses were performed to determine 
whether the theoretically-derived constructs measured by student survey items were internally 
consistent. Internal reliability coefficients were calculated for each subscale. Pre-test surveys 
from all students that completed both pre-tests and post-tests were used for this analysis. 

Exhibit 2 displays the characteristics of the subscales and the internal reliability of each subscale 
created from the student survey. As Exhibit 2 reveals, all of the subscales had moderately high 
reliability, with the exception of the civic knowledge measure, which included items assessing 
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both knowledge of government and communities. The civic knowledge score was normally 
distributed and positively correlated with student self-reports of grades. In the results section, 
subscale averages were calculated only from those item subsets with at least 50% valid 
responses. 



Exhibit 2. Characteristics of Pre-Test Student Survey Subscaies 



Subscale 


Mean or 
Average 


Standard 

Deviation 


Number of 
Items 


Range 


Internal 

Reliability 


Academic 


4.06 


.696 


4 


1-5 


.837 


Engagement 


Valuing School 


3.69 


.716 


5 


1-5 


.790 


School Attachment 


2.74 


.708 


7 


1-4 


.879 


Enjoyment of Math 


3.00 


1.079 


2 


1-5 


.508 


and Science 


Enjoyment of 


3.50 


1.012 


2 


1-5 


.639 


Reading/ 
Eanguage Arts 
and Social Studies 


Civic Knowledge 


2.48 


0.651 


5 


0-8 


.462 


Self-Reported Civic 


Knowledge 


2.48 


.651 


8 


1-4 


.878 


Civic Skills 


2.81 


.628 


5 


1-4 


.749 


Communitv 


Attachment 


2.50 


.776 


6 


1-4 


.884 


Civic Dispositions 


3.34 


.515 


5 


1-4 


.695 


Efficacy 


3.76 


.759 


3 


1-5 


.586 


Civic Engagement 


2.27 


.733 


6 


1-5 


.725 



Variabies That May Serve as Moderators of Outcomes 

This study also investigated a number of variables that, based on the literature review, were 
thought to be possible moderators of outcomes. Variables identified as possible moderators were 
characteristics of the service-learning experiences and characteristics of teachers and their 
practices. Characteristics of service-learning included duration, nature and type of service, and 
quality. Quality was measured in two ways: by asking students to rate various aspects of their 
experience on the post surveys and by asking teachers to describe and rate aspects of service- 
learning on the teacher survey. Qualitative information was also collected through interviews 
and focus groups. 

Specifically, the service-learning post surveys asked students about: 

• The quality of their experiences, that is, opportunities to reflect, make important decisions, 
develop and use their own ideas, feel that they had made a contribution, experience 
challenge, or experience adult criticism (reverse scored); 
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• The extent to which they believed that they had acquired academic and work-related skills as 
a result of their service-learning experience; 

• A measure of engagement in service-learning, e.g., the degree to which students worked hard 
on the service-learning project, enjoyed school more when working on service-learning, 
and/or whether they just acted as if they were working on service-learning (reverse scored). 

Principal components factor analysis with varimax rotation of the measure of self-ratings of 
gains in subject matters and other skills yielded three factors: 1) reading, writing and computer 
skills, 2) math and science skills, and 3) work-related skills. Exhibit 3 shows that internal 
reliabilities as for these scales were high. The overall service-learning engagement subscale was 
also highly reliable. Factor analysis of the measure of student engagement in service-learning 
yielded three subscales: service-learning engagement, school engagement, and work ethic. 
Exhibit 3 shows that the first and second subscales had high internal consistency, however, the 
third, which consisted of negatively worded items, did not and so was dropped from further 
analysis. 



Exhibit 3. Characteristics of Student Service-Learning Subscaies 



Subscale 


Mean or 
Average 


Number of 
Items 


Range 


Internal 

Reliability 


Service-learning quality 


3.74 


8 


1-5 


.844 


Perceived gains in reading, writing, and 


1.13 


3 


0-3 


.758 


computer 

Perceived gains in math/science skills 


0.45 


2 


0-2 


.689 


Perceived gains in work related skills 


1.81 


3 


0-3 


.674 


Overall service-learning engagement 


3.35 


11 


1-5 


.829 



Data from students with a pre-test, post-test, and parent permission to participate in the study 
were included in the analysis. The resultant data may be slightly skewed toward more positive 
results since students with chronic attendance problems or those that have dropped out may not 
be fully represented in the data. However, it is assumed that these sources of error were 
randomly distributed among the treatment and comparison sites, so the differences between 
groups should not have been affected. 

Teacher Surveys 

Teacher surveys were administered to service-learning and comparison teachers in the spring of 
2004. All teachers were asked about their field of certification, teaching experience, frequency 
of use of various instructional methods (e.g., lecture, debate, mock trials, cooperative learning, 
community service, etc.), coverage of civics topics, and perceived student growth in civic 
knowledge and skills. The service-learning teacher survey also included questions about teacher 
experience using service-learning as an approach, and the nature and quality of the teachers’ 
service-learning practice. 

Two techniques were developed to classify teachers’ self-reported instructional methods. Using 
subscales developed in prior research, RMC Research categorized the instructional methods 
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assessed in this study as aetive (a = .685 for this study) or passive (a = .480 for this study). 
Active teaching strategies required relatively high levels of student participation and included: 

• Community service or volunteering; 

• Visits to government or community institutions; 

• Debates or discussions; 

• Mock trials, role plays or other simulations; 

• Assignments in which students analyzed media presentations of information; 

• Research reports; 

• Student- generated projects; and 

• Cooperative learning. 



Passive teaching techniques included lectures; textbook reading; videos, DVDs, or television; 
and multiple choice tests. 

A cluster analysis was performed that resulted in three alternative ways to analyze data: 
subscales on traditional teaching techniques (a = .669), interactive techniques (a = .719), and 
product development techniques (a = .763). Traditional teaching techniques included lecture, as 
well as assessment techniques such as multiple choice tests, or essays. Interactive techniques 
provided opportunities for peer-peer or student-community interaction such as debates, working 
with the community, and cooperative learning. Product development techniques involved 
students in the creation of work products such as writing, videos, media, or mock trials. These 
subscales were used in a series of exploratory analyses. 

To examine the quality of service-learning, subscales were created by combining items on the 
teacher survey intended to measure the degree to which service-learning reflected the 1 1 
Essential Elements of Service-Learning (National Service-Learning Cooperative, 1998). Exhibit 
4 displays the descriptive results for these subscales. The Exhibit shows that the subscales varied 
with regard to internal reliability. Internal reliability could not be completed for service-learning 
elements measured with only one item. 
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Exhibit 4. Subscaies Designed to Measure Service-Learning Quaiity 



Number 







Mean or 


Standard 


of 




Internal 


Service-Learning Element 


Average 


Deviation 


Items 


Range 


Reliability 


1. 


Clear educational goals. 


3.64 


.285 


5 


1-4 


.183 


2. 


Involve students in cognitively challenging 
tasks. 


3.67 


.346 


3 


1-4 


.520 


3. 


Assessment used to enhance student 

learning and evaluate how well students 
have met content and skill standards. 


3.39 


.656 


2 


1-4 


.690 


4. 


Students are engaged in service tasks with 
clear goals that meet genuine community 
needs and have significant consequences. 


3.32 


.608 


4 


1-4 


.773 


5. 


Use of evaluation. 


3.07 


.997 


1 


1-4 


— 


6. 


Youth voice in selecting, designing, 
implementing, and evaluating service- 
learning projects. 


3.23 


.683 


4 


1-4 


.800 


7. 


Valuing diversity. 


3.86 


.363 


1 


1-4 


— 


8. 


Communication, interaction, partnerships, 
and collaboration with the community. 


3.20 


.508 


5 


1-4 


.719 


9. 


Students are prepared for all aspects of their 
service work. 


3.39 


.388 


5 


1-4 


.394 


10. 


Use of reflection. 


3.31 


.577 


6 


1-4 


.807 


11. 


Celebration and acknowledgment of service 
work. 


3.29 


.579 


2 


1-4 


.607 


Note 


!: Items range from 1 to 4 with 1 = Never or Almost Never; 2 = Sometimes; 3 = Often; 4 = Always or Almost Always. 



Data from the teacher survey allowed testing of several variables hypothesized to moderate 
outcomes including service-learning duration, nature, type, and quality; student characteristics; 
and teacher experience and practices. Differences between schools were also investigated. 

In addition to the surveys, each site was visited and qualitative data were collected in order to 
construct case studies. These methods are summarized along with the quantitative measures in 
Exhibit 5. Only quantitative results are presented here. 
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Exhibit 5. Summary of Data Coiiection Methods 





Student Surveys 


Teacher Surveys 




interviews 


Service-Learning 


Site 


Service- 

Learning 


Comparison 


Service- 

Learning 


Comparison 


Teacher Administrator 


Observations 


Student 

Focus 

Group 


Fort Myers, FL 


Academy 
N= 19 


Dunbar 

N=25 


N=4 


N=3 


N=4 


N= 1 


1 


N=6 


Anoka-Hennepin, 

MN 


Anoka 

N=60 


Coon Rapids 
A=49 


N=2 


N=2 


N=2 


N= 1 


1 


N= 15 


Miami-Dade, FL 


Homestead 
Senior 
N= 19 


Homestead 
Senior 
N= 13 


N= 1 


N= 1 


N= 1 


N= 1 


1 


N=1 




Miami Coral 
Park 
N= 134 


Miami Coral 
Park 
N= 103 


N= 1 


N= 1 


N= 1 


N= 1 


1 


N = 4 




Miami Lakes 
Educational 
Center 
N=99 


Miami Lakes 
Educational 
Center 
N=38 


N= 1 


N= 1 


N= 1 




1 


N=5 




South Miami 
N= 153 


South Miami 
N=57 


N=2 


N= 1 


N=2 


N= 1 


1 


N= 15 




William G. 

Turner 

Technical 

N=12 


William G. 

Turner 

Technical 

N=49 


N= 1 


N= 1 


N= 1 


N= 1 


1 


N= 13 


Humble, TX 


Quest 

A=34 


Humble 
N= 12 


N= 1 


N= 1 


N=3 


N=3 


1 


N= 15 


Tillamook/Rockaway 
Beach, OR 


Tillamook 

A=83 


Neah-Kah-Nie 

N=33 


N= 1 


N= 1 


N= 1 


N= 1 


1 


N =10 


TOTAL 


673 


379 


14 


12 


16 


10 


9 
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Results 



Overview 

The results are organized into three major seetions. The first seetion is a brief overview of 
student charaeteristies. The second section addresses the impact of service-learning over time by 
comparing participating students to their control group counterparts. Between group differences 
related to student perceptions of school and measures of civic development are analyzed for the 
whole sample and then for matched school pairs. The third section explores the roles of 
moderators in service-learning student outcomes. Potential moderators, including engagement, 
nature and type of service, duration of service, and the quality of the service-learning experience 
are examined for their relationships to outcomes for service-learning students. Finally, 
differences between service-learning and comparison teachers in civics content coverage and the 
use of active versus passive instructional strategies and the effects of these differences are 
discussed. 

Only students who completed both a pre-test and post-test were included in the analysis of 
survey results. A total of 1,052 students comprised the sample, 645 of whom were service- 
learning participants and 407 of whom were comparison group students. 

Student Characteristics 

Exhibit 6 presents a profile of the demographics of the sample. Nearly 60% of the sample was 
comprised of seniors and nearly 60% was female. The largest ethnic group represented was 
Hispanics, primarily from the Miami-Dade School District. Just over 60% of the sample spoke 
English at home. Most of the rest spoke Spanish. 
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Exhibit 6. Demographic Profiie of Students 
That Compieted Both Pre-Tests and Post-Tests 



Characteristic 


Number 


Percent 


Grade 


9 


146 


13.9 


10 


7 


0.7 


11 


260 


24.7 


12 


617 


58.7 


Not specified 


22 


2.1 


Gender 


Male 


419 


39.8 


Female 


627 


59.6 


Not specified 


6 


0.6 


Ethnicity^ 


White 


323 


30.8 


Asian/Pacific Islander 


16 


1.5 


Black/ African American 


135 


12.9 


American Indian/ Alaskan Native 


10 


1.0 


Hispanic/Latino 


572 


54.5 


Other 


54 


5.1 


Language Spoken at Home 


English 


662 


62.9 


Spanish 


329 


31.3 


Other 


36 


3.4 


Not specified 


25 


2.4 



^ Percentages do not sum to 1 00 because respondents could select more than 
one answer. 



Students were asked to report the grades they typically earned in classes. These self-reports are 
consistent with the achievement profiles for the schools. Over half of the students averaged 
“mostly Bs” with about a third of the sample averaging “mostly Cs” or below. About 18% 
reported “mostly As.” This profile represents an “average” national population, with a slight 
trend toward lower than average scores. 

About half of the students performed service in school before the academic year in which the 
study took place. More than 25% were volunteers with a youth organization and/or church. 
About 17% served the neighborhood, an indicator that service probably took place within the 
school. Less than 15% of the sample had no prior experience with service. 

Student Perceptions of School 

It was hypothesized that students that were engaged in service-learning would value and be more 
engaged in school than nonparticipating peers. To investigate this hypothesis, comparisons were 
made between the service-learning and comparison groups for the entire sample. Student ratings 
of the degree to which they valued school are presented in Exhibit 7. The results showed that 
students generally found school to be meaningful and important for later life sometimes or often. 
The ratings of school value declined over time for both groups. A repeated measures ANOVA 
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that compared the two groups revealed that there were only slight, non signifieant differenees 
between the service-learning and comparison groups on this subscale. 

Exhibit 7. Service-Learning and Comparison Student Ratings 

of the Vaiue of Schooi 




□ Pre-Test 

□ Post-Test 



Note: N = 1 ,046. Items range from 1 to 5 with 1 = Low Value, 5 = High Value. 



As illustrated in Exhibit 8, there was a small statistically significant difference between the 
serviee-learning and eomparison groups in sehool enjoyment, favoring the serviee-learning 
group. There were no differenees in ratings of enjoyment of speeifie subjeet matter. 
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Exhibit 8. Service-Learning and Comparison Student Ratings 
of Enjoyment of Schooi and Specific Content Areas 



3.73 



3.648 3.651 



3.531 



cs 

cc 

0) 3 



2.88a 



3.063 

2.973 3.073 3.0 






3.593 



^835 ^2-8 99 



o g>P 

E I g 

^ m II 

cn g " 



CO 



O 



Enjoy School 




CJ ? O) 

> .E ^ 

C c CD 

CD II 

CO g " 



.i g 

CO CO 
CL II 

o 



E I S 

CO g " 



~ CD 
CO 



a 



Enjoy Math 



Enjoy Science 



Item 



5 5? I 

; -p ^ CO 

D I ^ g- II 

® o e. 

o 

Enjoy Reading 



I rS 

If 
s " 

o 

Enjoy Social 
Studies 



Note: Items range from 1 to 5 with 1 = Never/Almost Never, 5 = Always/Almost Always. Enjoy school p < .05. 



Students responded to a number of other indicators that measured their affective, behavioral, and 
cognitive engagement in school. Exhibit 9 displays their overall responses on the subscale for 
Academic Engagement. This Exhibit shows that service-learning and comparison students rated 
their academic engagement higher at the pre-test than the post-test. Differences between groups 
were not statistically significant. 
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Exhibit 9. Service-Learning and Comparison Group Ratings 
of Academic Engagement {N= 1,050) 



5 



4.067 4.059 




Service-Learning (N = 672) Comparison (N = 378) 

Group 

Note: items range from 1 to 5 with 1 = Never/Aimost Never, 5 = Always/Aimost Aiways. 

Student Attachment to Schooi 

Exhibit 10 shows the overall differences in the subscale of School Attachment for service- 
learning participants and comparison students. The service-learning group had higher overall 
attachment to school than the comparison group in the fall and spring; however, the differences 
were not statistically significant. Declines over time may reflect the fact that a large percentage 
of students in the sample were seniors and were leaving the school upon graduation in the next 
few weeks after the survey was administered. 
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Exhibit 10. Service-Learning and Comparison Student Ratings 
of Attachment to Schooi {A/= 1,046) 




Group 



□ Pre-Test 
n Post-Test 



Note: Items range from 1 to 4 with 1 = Not At All, 4 = A Lot. 



Students were asked to respond to a set of questions about their community attachment that 
paralleled the questions on attachment to school. As can be seen in Exhibit 11, the service- 
learning and comparison groups responded at the midpoint between “a little” and “some” at the 
pre-test, and their ratings declined slightly over time. The decrease for the service-learning 
group was somewhat smaller than the comparison group, but the difference was not statistically 
significant. 



The Impact of Participation in Service-Learning on 20 
High School Students’ Civic Engagement 



RMC Research Corporation, Denver, CO 



Exhibit 11. Service-Learning and Comparison Student Ratings of 
Community Attachment {N= 1,046) 





Group 



Note: Items range from 1 to 4 with 1 = Not At All, 4 = A Lot. 

Measures of Civic Engagement 

Two measures of civic knowledge were administered. Students were asked a number of factual 
questions about government and civics. As illustrated in Exhibit 12, both the service-learning 
and comparison groups scored higher on the post-survey. Differences were not significant across 
groups. 



Exhibit 12. Service-Learning and Comparison Student Scores on 
Civic Knowiedge Questions {N= 1,051) 




□ Pre-Test 

□ Post-Test 



Group 

Note: Scale reflects total questions correct out of 8. 
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Students were also asked to identify the names of the two United States Senators serving then- 
state and to identify the three branehes of government. Students generally did not know the 
names of the two United States senators serving their state. On the post-test for this question, 
serviee-learning students seored signifieantly higher than their peers. Students in both groups 
typieally knew one or two of the branehes of government, and the eomparison group performed 
significantly better over time on this question. 

Students were asked to identify the year in which the next presidential election was to be held. 
About two thirds of the students in both groups knew when the next Presidential election was 
scheduled and both improved over time, probably because the next election was only months 
away and news about the election was prominent in the media. About a third of each group 
could name two community service organizations in their city or town at both the pre- and post- 
test. Less than 20% could provide an example of how the United States government limits the 
power of the government at the pre-test. This increased slightly at the post-test for both groups, 
with a larger increase for the comparison group than the service-learning group. The differences 
for each of these items were not statistically significant. 

On a second measure of civic knowledge, students were asked to rate themselves on a series of 
items that measured the degree to which they felt well informed about the ways in which society 
and government works. Results for the self-ratings are presented in Exhibit 13. The comparison 
group reported being better informed than the service-learning group, while the service-learning 
group demonstrated greater gains over time. These differences were not statistically significant. 

Exhibit 13. Service-Learning and Comparison Student Ratings 
of Their Civic Knowiedge {N=^ ,042) 




Group 

Note: Items range from 1 to 4 with 1 = Not At All Informed, 4 = Very Well Informed. 



Both groups reported feeling better informed over time about how to register to vote; how local 
state, and federal government decisions are made; the difference between democracy and 
socialism; the difference between how decisions are made in the United States and in Great 
Britain; how interest groups locally operate; and how the Electoral College works. No 
statistically significant differences between service-learning participants and comparison 
students were found for these items. Both groups increased slightly over time in the reported 
intention to vote when they are of age. 
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Students were asked to respond to a series of questions designed to measure civie dispositions 
including their attitudes toward civic responsibility, and intent to participate in civic and political 
life. Service-learning students had slightly higher scores on this measure than comparison 
students at both points in time. Exhibit 14 shows that the service-learning group’s civic 
dispositions remained the same over time, while the comparison group increased slightly. More 
generally, the data show that students tend to be favorably disposed to being civically engaged. 



Exhibit 14. Service-Learning and Comparison Student Ratings 
of Civic Dispositions {N= 1,043) 



3.362 3.365 



3.292 3.33 



cd 

CC 

<D 

O) 

CO 

0 2 
< 





Service-Learning (N = 669) Comparison (N = 374) 

Group 



Note: Items range from 1 to 4 with 1 = Definitely Disagree, 4 = Definitely Agree. 



Most students in both groups felt that doing something about community problems is a job for all 
young people and that being informed about state and especially local issues is an important 
responsibility for everyone. Fewer students saw the importance of being actively involved in 
community issues. Doing things to help improve America was seen as very important to nearly 
all of the students. Students showed a slight tendency to agree that being actively involved in 
political issues was not that important, but generally reported that they intended to vote and 
intended to volunteer when they got older. The difference between the two groups on intention 
to vote was statistically significant, favoring the service-learning group. Interestingly, about 
15% of students said they might run for political office sometime in the future. Most agreed they 
were interested in the news and current events. Most said that they did not know how to conduct 
an election campaign. There were no statistically significant differences on the last three items. 

Students rated themselves on a number of civic skills, such as leading other students and 
conducting a campaign. As shown in Exhibit 15, differences between the service-learning 
students and comparison students were very small but slightly favored the service-learning 
students. In the analysis of specific skills items, group differences were found for those that 
pertained most directly to service-learning. Over time, the service-learning participants were 
more likely to report that they knew how to work with others to solve a community problem and 
that they knew how to identify community needs. However, these differences were not 
statistically significant. 
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Exhibit 15. Service-Learning and Comparison Student Ratings 
of Civic Skiiis {N= 1,043) 




□ Pre-Test 

□ Post-Test 



Group 

Note: Items range from 1 to 4 with 1 = Definitely Disagree, 4 = Definitely Agree. 



Students were asked about the frequency with which they engaged in a variety of activities 
related to civic life, such as discussing politics with friends, families, or teachers, participating in 
rallies or going to political or civics lectures, and following the news. Responses indicated that 
students averaged between “seldom” and “sometimes” on most activities. Differences between 
groups were not statistically significant, as shown in Exhibit 16. Results showed that students 
sometimes discussed politics. Political discussion occurred more often in the spring of 2004 than 
the fall of 2003, probably due to the Presidential elections that occurred in November 2004. Few 
students attended rallies, and this percent decreased over time for both groups. Still fewer wrote 
letters expressing an opinion to a public official. About two thirds of the sample said that they 
never or almost never did this. Nearly the same percentage never or almost never refused to 
purchase a product because of where it was made, though ratings on this item increased slightly 
over time. Environmental protection and taking care of the needy were more common activities 
for this sample of students though ratings on helping the needy declined for both groups over 
time. 
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Exhibit 16. Students’ Ratings of Participation 
in Civic Engagement Activities (Af= 1,039) 



5 



4 



O) 

■fo 

CC 




Service-Learning (N = 667) Comparison {N = 372) 

Group 



□ FVe-Test 
■ Rest-Test 



Note: Items range from 1 to 5 with 1 = Never/Almost Never, 5 = Always/Almost Always. 



A measure of student efficaey was created using how often students felt that they made a 
difference in someone’s life and took on adult responsibilities. It was hypothesized that those 
students that engaged in service-learning would increase their ratings on this subscale while 
comparison students would not. The data, displayed in Exhibit 17, showed no differences 
between the service-learning and comparison groups on the efficacy subscale. Both groups felt 
that they sometimes made a lot of difference in people’s lives, and both groups reported 
increases in having adult responsibilities. The latter result might be related to the large numbers 
of seniors in the sample that felt a need, with graduation, to assume more adult roles. The 
absence of any difference in students’ sense that they were making a difference may be 
explained by service-learning students’ having taken on issues that were so daunting that 
students felt they could not make a difference. Alternatively, the general wording of this item 
may have made it inadequate for assessing perceived impact on the specific recipients of service. 
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Exhibit 17. Service-Learning and Comparison Student Ratings 

of Efficacy (/V= 1,036) 




Group 



Note: Items range from 1 to 5 with 1 = Never/Almost Never, 5 = Always/Almost Always. 

Differences Between Service-Learning and Comparison Group Outcomes For 
Matched Schooi Pairs 

The preceding section showed differences for the overall study sample and did not take into 
account exposure to specific subject matters or grade levels of the students. The design for this 
study called for matched classrooms to account for these differences. This section details the 
results of hierarchical tests of statistical significance of differences between matched groups of 
students at the same or similar schools in classrooms that covered the same content areas. 

Aggregated results of these comparisons revealed statistically significant differences in the 
change scores between pre-test and post-tests for service-learning and comparison students on 
three subscales: Civic Knowledge, Civic Dispositions, and Valuing School. Several within- 
school pair differences accounted for the significant differences overall. 

• Statistically significant within-school pair differences were found in scores of factual civic 
knowledge at four sites, all of which had high quality service-learning programs (discussed 
below). Service-learning students at three sites were found to have made greater gains in 
civic knowledge than comparison students; while comparison students at one site where the 
service-learning program was weak showed greater improvement. 

• Significant group differences in gains in civic dispositions occurred at one site with a weak 
service-learning program, with comparison students showing greater increases. 
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• On the Valuing School subscale, significant between-group differences occurred at one site 
that had a very weak service-learning program. For this pair, comparison students made 
greater gains than service-learning students. 

Repeated measures hierarchical analyses of variance were also conducted on the individual 
survey items and differences were found for seven specific items. For each of these statistically 
significant item level results, follow-up mixed analyses of variance (ANOVA) were conducted to 
determine which school pairs were associated with the significant results. The effect sizes were 
in the low to moderate range for all of the differences that were found. 

Moderators of Impact 

The literature and the analysis of paired groups of students suggests that the lack of differences 
between service-learning and comparison students may be due to several moderating factors, 
including the extent to which students were engaged in the service-learning experience; the 
duration, nature, type, and quality of the service-learning experience; and the extent to which the 
service experience was linked to content coverage. In addition, studies suggest that teacher 
experience, expectations, and use of active learning strategies may also serve as moderating 
factors. This section examines the extent to which each of these variables influenced the 
outcomes that were found. The effects of aspects of the service-learning experience are 
summarized first, followed by an examination of the degree to which teacher characteristics and 
practices served as moderators of impact. Differences between groups are shown to illuminate 
the strong variation in perceived quality and the impact of that variation on outcomes. 

Students’ Service-Learning Experiences 

Data showed that there were wide variations in the ways in which service-learning was 
conducted at the various schools. At three schools, all students completed year-long projects that 
were closely linked to class goals and explicitly focused on skills needed for effective 
democratic engagement, such as problem analysis, collaborative problem solving, civic dialogue, 
and the ability to work within the political system to improve society. For example, at one 
school, service-learning was part of a year-long Senior Seminar in which students conducted 
inquiry into global problems and then identified a “social problem they (were) passionate about” 
based on the results of their research. Students worked in groups to develop sustainable action 
plans to address the problem they chose. Action plans were required to state the ways in which 
students would increase others’ consciousness of the problem and include solutions with lasting 
impact. Students worked together to implement their plans and presented the results in a 
multimedia presentation and portfolio. 

At another school, service-learning took the form of a year-long civic problem-solving project. 
Students generated a list of local, state, and national needs. Teams of students then selected a 
need and conducted research on it, including interviews with elected officials. They wrote 
proposals for solutions and organized presentations to the public where they presented their 
research and potential solutions. One team of students at this school sought to address the issue 
of traffic congestion in the city. After conducting research on solutions that had been developed 
in other metropolitan areas, students developed a proposal for a rapid transit system that they 
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presented to state offieials. A seeond team studied the national problem of methamphetamine 
abuse. After investigating national incidence rates, causes, and consequences of this abuse, they 
sponsored a forum where students, civic leaders, and community members together developed a 
strategic plan for addressing the problem locally. 

Social studies students at another site organized a public forum in preparation for their city’s 
impending mayoral election. Students interviewed members of different segments of the 
community to learn about their concerns and wrote interview questions for mayoral candidates 
based on their research. They issued invitations to candidates, recruited a local news announcer 
as a moderator, made the necessary arrangements, and hosted the forum. In follow-up class 
discussions, students evaluated candidates’ positions and the quality of their presentations. 

Other projects observed in the study were extensive but not directly designed to build civic s ki lls. 
For example, in one program, students in environmental studies, art, and video production 
classes worked much of the year to transform a debris-filled lot into a park complete with flower 
beds and a bench. Finally, some student projects observed in this study involved limited time 
and/or student participation. At one school, for example, students learned about Social Security 
law in preparation for an intergenerational forum sponsored by their district. However, 
participation in the project was voluntary and primarily involved research, rather than 
community activity. At another school that completed a Social Security project, the service 
experience only spanned a few-week period. 

All students that participated in service-learning activities were asked to respond to questions on 
the spring post- test survey about their service-learning experiences. Exhibit 18 displays the 
results of the survey. The Exhibit shows that 83% of the students felt that they often or almost 
always had real responsibilities, 76% had challenging tasks, and 77% made important decisions. 
Most of the students also felt that they often or almost always had freedom to develop and use 
their own ideas (68%), did things themselves rather than observing (62%), and felt they made a 
contribution (58%). While about 43% often or almost always discussed experiences with their 
teachers, nearly one third felt often or almost always that adults criticized them or their work. 

The latter indicators showed the degree to which students felt valued and were treated 
respectfully by adults. These items are indicators of the quality of the service-learning 
experience, and these data reveal that about two thirds of the students had a relatively high 
quality service-learning experience using the definition of quality extant in the field. 
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Exhibit 18. Student Ratings of Their Service-Learning Experiences 



Average 

Rating Percent Distribution of Responses 

Never/ Always/ 

Almost Almost 





N 


Mean 


S[T 


Never 


Seldom 


Sometimes 


Often 


Always 


Had real 
responsibilities. 


642 


4.24 


.892 


1.4 


3.4 


12.0 


36.4 


46.7 


Had challenging 
tasks. 


640 


4.07 


.932 


1.6 


4.2 


18.1 


38.0 


38.1 


Made important 
decisions. 


636 


4.08 


.952 


2.0 


3.9 


17.5 


37.1 


39.5 


Discussed my 
experience(s) 
with teacher(s). 


639 


3.23 


1.291 


11.3 


19.6 


25.5 


22.5 


21.1 


Did things myself 
instead of 
observing. 


638 


3.81 


.976 


1.9 


5.8 


30.3 


33.7 


28.4 


Had freedom to 
develop and use 
my own ideas. 


641 


3.93 


.981 


1.9 


5.5 


24.2 


34.8 


33.2 


Adults criticized 
me or my work. 


637 


2.93 


1.291 


16.0 


22.8 


29.7 


15.4 


16.2 


Felt I made a 


639 


3.69 


1.073 


3.4 


9.5 


28.3 


31.8 


26.9 



contribution. 

® Standard deviation. 

Exhibit 19 shows students’ perceptions of the academic and work-related skills they obtained 
from their service-learning experiences. About two thirds of the service-learning students felt 
they gained job skills and work experience and over half said they became more aware of 
careers. Over 40% believed they gained reading skills, and over a third said they gained writing 
skills and computer skills. About a fourth said they learned math skills and tutoring skills. 
Finally, slightly fewer than 20% of students said they learned science skills. These figures are 
consistent with their ratings of enjoyment and skill acquisition in particular content areas from 
the first half of the survey. 
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Exhibit 19. Student Perception of Skiiis Acquired 
From Service-Learning (Af = 645) 



Skill Area 


Percent Reporting Gain 


Reading skills 


41 


Writing skills 


38 


Math skills 


26 


Science skills 


19 


Computer skills 


35 


Tutoring skills 


26 


Career awareness 


51 


Job skills 


66 


Work experience 


64 



As discussed previously, students’ ratings on this measure clustered into three areas: reading, 
writing and computer skills; math and science skills; and work-related skills. Perceived learning 
about tutoring emerged as a single item that was unrelated to their responses to other items. A 
one-way MANOVA of the three subscales and the tutoring item showed significant overall 
between-program differences in perceived skill learning F(32, 2,544) = 4.693, p < .001, Eta- 
squared = .056. Univariate ANOVAs were then conducted using each skill cluster as a 
dependent measure. Results of these ANOVAs, which are displayed in Exhibit 20, were highly 
statistically significant. 

Exhibit 20. Differences Between Schoois in Perceived Skiii Learning 

Spring 2004 



Skill 


Sum of 
Squares 


df 


Mean 

Square 


F 


P 


Partial 

Eta- 

squared 


Reading, Writing, and Computer 
Skills 


45.725 


8 


5.716 


4.174 


.000 


.050 


Mathematics and Science Skills 


15.438 


8 


1.930 


3.787 


.000 


.045 


Tutoring Skills 


11.513 


8 


1.439 


8.085 


.000 


.092 


Career, Job, and Work Skills 


45.569 


8 


5.696 


4.648 


.000 


.055 



Student Engagement in Service-Learning 

Students also responded to a series of items designed to measure their engagement in service- 
learning activities. A majority of students said that they often or almost always worked very 
hard on their service-learning projects, paid attention when planning or working on service- 
learning activities, and tried to learn as much as they could about the service-learning project. 
Just over half said that it was often important or almost always important to them to do the best 
they could when working on their service-learning project. Over 40% said they show up to 
school often or almost always when working on their service-learning projects, and nearly the 
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same percentage said they often or almost always do their homework more often and enjoy 
school more. Results are presented in Exhibit 21. 

Exhibit 21. Student Engagement in Service-Learning 



Average 

Rating Percent Distribution of Responses 

Never/ Aiways/ 

Almost Aimost 





N 


Mean 


SD" 


Never 


Seldom 


Sometimes 


Often 


Always 


I work very hard on my 
service-learning 
project. 


634 


3.71 


1.101 


4.4 


10.6 


20.8 


37.7 


26.5 


I try to learn as much 
as I can about my 
service-learning 
project. 


633 


3.65 


1.113 


4.3 


12.2 


23.7 


34.4 


25.4 


When I am doing 
service-learning, I try 
very hard. 


630 


3.81 


1.092 


3.7 


9.5 


20.6 


34.9 


31.3 


When I am doing 
service-learning, I 
just act as if I am 
working. 


624 


2.83 


1.338 


15.5 


16.7 


21.5 


27.4 


18.9 


I pay attention when 
planning or working 
on my service- 
learning activities. 


630 


3.92 


1.160 


3.3 


7.3 


19.9 


34.6 


34.9 


I don’t work very hard 
at service-learning. 


629 


3.91 


1.085 


39.8 


31.4 


15.1 


8.6 


5.1 


It is very important to 
me to do the best I 
can when working on 
my service-learning 
project. 


626 


3.49 


1.284 


9.7 


13.1 


23.3 


26.4 


27.5 


When I participate in 
service-learning, I 
show up to school on 
time more often. 


623 


3.22 


1.243 


11.2 


16.7 


29.1 


24.9 


17.8 


When I participate in 
service-learning, I 
enjoy school more. 


628 


2.93 


1.436 


25.3 


13.1 


23.1 


20.7 


17.8 


When I participate in 


627 


3.08 


1.288 


15.6 


16.3 


28.4 


23.8 


15.9 



service-learning, I do 
my homework more 
often. 

® Standard deviation 
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student Engagement in Service-Learning as a Moderator of Student Outcomes 

The study hypothesized that service-learning would lead to greater student engagement which in 
turn would generate increased student outcomes. To test this hypothesis, a canonical correlation 
analysis was used. The overall engagement scale score was significantly related to the student 
outcome measures F{\2, 593) = 37.598, p < .001, Eta-squared = .432. Univariate analyses 
indicated that student engagement was significantly associated with every outcome measure 
except one (civic knowledge). Exhibit 22 displays the results for the overall service-learning 
engagement score and its relationships to each of the student outcome measures. 

Exhibit 22. Service-Learning Engagement as a Moderator of Student Outcomes 



Dependent Variable 


F 


P 


Partial Eta- 
squared 


Academic Engagement 


200.901 


.000 


.250 


Valuing School 


175.162 


.000 


.225 


School Attachment 


130.944 


.000 


.178 


Enjoyment of Math and Science 


25.391 


.000 


.040 


Enjoyment of Reading/Eanguage 
Arts and Social Studies 


59.232 


.000 


.089 


Civic Knowledge 


.070 


.792 


.000 


Self-Reported Civic Knowledge 


23.592 


.000 


.038 


Civic Skills 


109.269 


.000 


.153 


Community Attachment 


108.774 


.000 


.153 


Civic Dispositions 


80.811 


.000 


.118 


Efficacy 


175.764 


.000 


.225 


Civic Engagement 


103.500 


.000 


.146 



Principal component factor analysis with varimax rotation yielded three subscales within the 
student engagement scale: engagement in service-learning activities, engagement in school due 
to service-learning, and work ethic while engaged in service-learning. Differences between 
service-learning schools on these factors were analyzed using a multivariate analysis of variance 
(MANOVA) with school as the independent measure. Results of the MANOVA were highly 
significant F(24, 1,863) = 4.972, p < .001, Eta-squared = .060. Univariate tests of between- 
school differences on each factor were highly significant for service-learning engagement F(8, 
621) = 3.929, p < .001, Eta-squared = .048 and for engagement in school due to service-learning 
F(8, 621) = 5.278, p < .001, Eta-squared = .064. A MANOVA was conducted to find between- 
school differences in each item on the engagement scale. Results of the MANOVA were highly 
significant E(88, 4,672) = 2.672, p < .001, Eta-squared = .048. Univariate analyses revealed 
significant differences across programs on all engagement items except one. Exhibit 23 displays 
the results of these analyses. 
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Exhibit 23. Differences Between Program for Service-Learning 
Student Engagement Ratings, Spring 2004 



Finally, please tell us how often 








Partial 


you feel the following statements 








Eta- 


are true for you. 


df 


F 


P 


squared 


I work very hard on my service-learning 
project. 


8 


3.974 


.000*** 


.051 


I try to learn as much as I can about my 
service-learning project. 


8 


3.624 


.000*** 


.047 


When I am doing service-learning, I try 
very hard. 


8 


4.785 


.000*** 


.061 


When I am doing service-learning, I just 
act as if I am working, (reversed item) 


8 


1.032 


.410 


.014 


I pay attention when planning or working 
on my service-learning project. 


8 


3.165 


.002** 


.041 


I don’t work very hard at service-learning 
(reversed item) 


8 


2.856 


.004** 


.037 


It is very important to me to do the best I 










can when working on my service- 
learning project. 


8 


2.448 


.013* 


.032 


When I participate in service-learning, I 
show up to school on time more often. 


8 


3.230 


.001** 


.042 


When I participate in service-learning, I 
enjoy school more. 


8 


6.311 


.000*** 


.079 


When I participate in service-learning, I 
disrupt class less. 


8 


2.173 


.028* 


.029 


When I participate in service-learning, I 
do my homework more often. 


8 


3.489 


.001** 


.045 



Nature and Type of Service as Moderators of Student Outcomes 

This study also explored whether the nature and type of service-learning activities impacted the 
outcomes. Service-learning experiences were classified according to two dimensions: (1) the 
nature of the issue addressed by the service project (e.g., environmental, civic, or varied based on 
student choice) and (2) the type of service performed by students (direct, indirect, or civic and 
political action). Exhibit 24 displays the classification scheme used to analyze the nature of the 
service-learning issue. 



RMC Research Corporation, Denver, CO 



33 



The Impact of Participation in Service-Learning on 
High Schooi Students’ Civic Engagement 



Exhibit 24. Nature of Student Service Issue 



Nature of Service Examples 

Environment Fixing a nature trail 

Testing water quality and reporting results 
Petitioning a city commission to start a recycling center 
Civic issue Working on the election 

Researching a public policy issue 
Varied (student selected) Tutoring 

Assisting an ill person 

Creating health-related educational materials 
Conducting oral history project with seniors 
Writing educational materials about senior abuse 
Serving at a soup kitchen 



The hypothesis tested was that students had more control over their own learning, that is, those 
that had a voice in selecting the community problem to be addressed, designed the service, and 
possibly developed an assessment of the impact of the service, would experience higher degrees 
of impact than those that were directed by their teachers. Further, those that provided service in 
an area directly related to civics were expected to show greater gains than those that engaged in 
service in a field not directly related to civics, such as the environment. 



A one-way MANOVA was conducted to determine the effects of the nature of the service issue 
on participants’ overall post- test scores. Results were statistically significant F(24, 1,244) = 
5.593, p < .001, Eta-squared = .097. Descriptive statistics are displayed in Exhibit 25. 



Exhibit 25. Effect of Nature of Service Issue on 
Students’ Civic Knowiedge and Community Attachment 



Outcome 


Nature of Service 


N 


Mean 


SD 


Civic Knowledge 


Environmental 


14 


1.86 


1.167 




Civic issue 


522 


2.83 


1.596 




Varied (student selected) 


99 


3.05 


1.886 


Self-Reported Civic Knowledge 


Environmental 


14 


2.03 


.643 




Civic issue 


522 


2.70 


.667 




Varied (student selected) 


99 


2.78 


.645 


Community Attachment 


Environmental 


14 


2.67 


.795 




Civic issue 


522 


2.39 


.787 




Varied (student selected) 


99 


2.67 


.855 



Post hoc tests indicated statistically significant differences for three outcomes: civic knowledge, 
self-reported civic knowledge, and community attachment. Students that chose their service 
activity made significantly greater gains in civic knowledge than those that focused on a teacher- 
selected civic problem (p < .020) or an environmental issue ip < .000). Students that addressed a 
teacher-selected civic issue made greater gains than those whose projects focused on an 
environmental problem ip < .025). Eor self-reported civic knowledge, students whose teachers 
selected civic issues performed better than students that selected topics themselves ip < .004). 
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Both of these groups exceeded those students that worked on the environment. Finally, students 
that chose their service projects rated themselves as more attached to the community than 
students that worked on the environment. Both of these groups had higher community 
attachment scores than those that worked on civic issues. 

The study also investigated the hypothesis that the type of service activity performed by students 
would serve as a moderating variable. To test this hypothesis, categories of service were 
established for projects that involved direct contact with service recipients, indirect service, and 
service involving civic or political action. Exhibit 26 displays the categorization scheme for type 
of service activity. Direct contact with service recipients was expected to result in higher 
outcomes, and service involving civic or political action was hypothesized to produce stronger 
civic outcomes. The MANOVA analysis of effects on the post-test outcomes was significant 
F(24, 1,028) = 5.326, p < .001, Eta-squared = .111. In follow-up univariate tests, service type 
was significantly related to scores on nearly the entire set of subscales, as shown in the effect 
sizes in Exhibit 27. 



Exhibit 26. Type of Service Activity 



Type 


Activity 


Direct 


Activities involving direct contact with recipients 
of service, e.g., tutoring, visits with seniors. 


Indirect 


Activities intended to benefit recipients with no 
direct contact, e.g., fundraising, research. 


Political or civic action 


Activities intended to influence political 

institutions or processes, e.g., circulating a 
petition, organizing a community forum. 



Exhibit 27. Reiationship Between Types of Service Activity 
(Direct, Indirect, Civic, or Poiiticai Action) and Student Outcomes 



Post-Test Student Outcomes 


df 


F 


P 


Partial Eta-squared 


Academic Engagement 


2 


4.761 


.009 


.018** 


Valuing School 


2 


4.082 


.017 


.015* 


School Attachment 


2 


1.825 


.162 


.007 


Enjoying Math and Science 


2 


5.810 


.003 


.022** 


Enjoying Reading/Eanguage Arts and 
Social Studies 


2 


13.000 


.000 


Q4§*H=* 


Civic Knowledge 


2 


13.612 


.000 


Q4g*H=* 


Self-Reported Civic Knowledge 


2 


9.577 


.000 


.035*** 


Civic Skills 


2 


0.549 


.578 


.002 


Civic Dispositions 


2 


5.921 


.003 


.022** 


Community Attachment 


2 


3.705 


.025 


.014* 


Efficacy 


2 


1.994 


.137 


.008 


Civic Engagement 


2 


0.310 


.734 


.001 


* p < .05, two-tailed test; ** p < .01 , two-tailed test, 


*** p< 


.001 , two-tailed test. 
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Exhibit 28 provides descriptive data for significant differences, showing that: 



• Indirect service was associated with higher levels of academic engagement, valuing school, 
and enjoyment of specific subject matters than direct service or civic or political action. 

• Students that participated in civic or political action had higher post-test civic knowledge and 
self-reported civic knowledge scores than those involved in indirect or direct service. 

• Students that engaged in civic or political action scored higher than others on civic 
dispositions. 

• Students that performed direct service had higher scores on community attachment. 

Exhibit 28. Types of Service and Student Outcomes 



Outcome 


Type of Service 


N 


Mean 


Standard 

Deviation 


Academic Engagement 


Direct 


12 


3.94 


.820 




Indirect 


96 


4.11 


.611 




Civic or political action 


419 


3.86 


.722 


V aluing School 


Direct 


12 


3.76 


.735 




Indirect 


96 


3.84 


.643 




Civic or political action 


419 


3.62 


.685 


Enjoyment of Math and 


Direct 


12 


3.04 


.925 


Science 


Indirect 


96 


3.34 


.859 




Civic or political action 


419 


2.94 


1.082 


Enjoyment of 


Direct 


12 


3.21 


1.033 


Reading/Eanguage Arts 


Indirect 


96 


3.96 


.850 


and Social Studies 


Civic or political action 


419 


3.45 


.927 


Civic Knowledge 


Direct 


12 


1.67 


1.155 




Indirect 


96 


2.21 


1.353 




Civic or political action 


419 


3.01 


1.626 


Self-Reported 


Direct 


12 


2.13 


.630 


Civic Knowledge 


Indirect 


96 


2.48 


.739 




Civic or political action 


419 


2.73 


.653 


Civic Dispositions 


Direct 


12 


3.00 


.553 




Indirect 


96 


3.31 


.556 




Civic or political action 


419 


3.43 


.504 


Community Attachment 


Direct 


12 


2.71 


.838 




Indirect 


96 


2.55 


.740 




Civic or political action 


419 


2.37 


.793 



Duration of Service-Learning as a Moderator of Student Outcomes 

The study investigated the hypothesis that duration of service-learning was associated with 
student outcomes. Teachers estimated the length of time their classes spent on all activities 
associated with the service-learning project, including the service activity itself and the steps of 
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preparation, reflection, and celebration/demonstration. Estimates ranged from less than one 
month to one academic year. 

A one-way MANOVA revealed statistically significant effects for the duration of the service- 
learning experience on students’ overall post-test scores F(36, 1,866) = 6.326, p < .001, Eta- 
squared = .109. Univariate tests showed that duration significantly influenced changes in ratings 
on: academic engagement, valuing school, school attachment, enjoyment of math and science, 
enjoyment of reading/language arts and social studies, civic knowledge, self-reported civic 
knowledge, and community attachment. Exhibit 29 shows the relationship between duration and 
each post-test outcome. Details of the statistically significant relationships between duration of 
the service-learning experience and student outcomes may be found in Exhibit 30 for all areas in 
which statistically significant relationships were found. The Exhibit shows that academic 
engagement, valuing school, and enjoyment of subject matter scores were highest when projects 
were one or two months in duration. School attachment, civic skills, and community attachment 
were rated highest by students in programs of 1 semester in duration. Civic knowledge was 
highest when projects were a year in duration. 

Exhibit 29. Effect Sizes for Service-Learning Duration and Student Outcomes 



Post-Test Student Outcomes 


df 


F 


P 


Partial 

Eta- 

Squared 


Academic Engagement 


3 


4.283 


.005 


.020** 


Valuing School 


3 


5.058 


.002 


.023** 


School Attachment 


3 


15.041 


.000 


.067*** 


Enjoying Math and Science 


3 


6.511 


.000 


.030*** 


Enjoying Reading/Eanguage Arts and Social 
Studies 


3 


15.018 


.000 


.067*** 


Civic Knowledge 


3 


13.062 


.000 


.058*** 


Civic Knowledge Self-Ratings 


3 


6.379 


.000 




Civic Skills 


3 


2.666 


.047 


.013* 


Civic Dispositions 


3 


1.027 


.380 


.005 


Community Attachment 


3 


9.620 


.000 


Q44* * * 


Efficacy 


3 


0.800 


.494 


.004 


Civic Engagement 


3 


2.193 


.088 


.010 



* p < .05, two-tailed test; ** p < .01 , two-tailed test, *** p < .001 , two-tailed test. 



RMC Research Corporation, Denver, CO 



37 



The Impact of Participation in Service-Learning on 
High School Students’ Civic Engagement 



Exhibit 30. Duration of Project and Student Outcomes 



Standard 



Outcome 


Type of Service 


N 


Mean 


Deviation 


Academic Engagement 


Eess than 1 month 


127 


3.85 


.772 




1 to 2 months 


112 


4.08 


.613 




1 semester 


59 


3.92 


.710 




1 year 


337 


3.80 


.776 


Valuing School 


Eess than 1 month 


127 


3.58 


.691 




1 to 2 months 


112 


3.83 


.627 




1 semester 


59 


3.62 


.667 




1 year 


337 


3.53 


.743 


School Attachment 


Eess than 1 month 


127 


2.37 


.839 




1 to 2 months 


112 


2.67 


.687 




1 semester 


59 


3.12 


.491 




1 year 


337 


2.73 


.734 


Enjoying Math and 


Eess than 1 month 


127 


2.97 


1.133 


Science 


1 to 2 months 


112 


3.30 


.919 




1 semester 


59 


3.06 


1.103 




1 year 


337 


2.80 


1.058 


Enjoying 


Eess than 1 month 


127 


3.16 


.984 


Reading/Eanguage Arts 


1 to 2 months 


112 


3.95 


.831 


and Social Studies 


1 semester 


59 


3.28 


.939 




1 year 


337 


3.53 


.962 


Civic Knowledge 


Eess than 1 month 


127 


2.62 


1.587 




1 to 2 months 


112 


2.54 


1.409 




1 semester 


59 


2.57 


1.367 




1 year 


337 


2.81 


1.696 


Civic Knowledge Self- 


Eess than 1 month 


127 


2.62 


.738 


Ratings 


1 to 2 months 


112 


2.54 


.710 




1 semester 


59 


2.57 


.625 




1 year 


337 


2.81 


.621 


Civic Skills 


Eess than 1 month 


127 


2.76 


.684 




1 to 2 months 


112 


2.85 


.694 




1 semester 


59 


3.04 


.480 




1 year 


337 


2.87 


.614 


Community Attachment 


Eess than 1 month 


127 


2.19 


.816 




1 to 2 months 


112 


2.50 


.763 




1 semester 


59 


2.85 


.579 




1 year 


337 


2.44 


.817 



Quaiity of the Service-Learning Experience as a Moderator of Student Outcomes 

In concert with the research, this study hypothesized that service-learning classrooms that were 
rated highly on various quality indicators were much more likely to yield positive outcomes. 
Teachers that implemented service-learning were asked to complete questions about various 
components of their service-learning projects that the literature in the field of service-learning 
indicates are associated with quality. Results showed that teachers perceived that their practices 
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were generally eonsistent with these indieators. The highest ratings were for having students 
communicate information, requiring students to apply academic concepts and/or content that 
they learned, implementing cognitively challenging service-learning tasks, meeting genuine 
community needs, and promoting higher order thinking skills. In addition, teachers felt that 
service-learning tasks helped students develop self-confidence, that there were clear educational 
goals for the service-learning projects, and that service-learning helped students develop personal 
efficacy. Students were perceived as playing a large role in implementing service-learning 
projects, but less of a role in designing, selecting, or evaluating service-learning. Scores were 
lowest for students’ having knowledge of community resources, receiving recognition for their 
service work from the community, establishing clear roles among partners in the service-learning 
project, and having an impact on the school. 

A composite score of service-learning quality was created by averaging all 38 responses from the 
teacher survey. A multiple linear regression using this total service-learning quality score to 
predict student outcomes was statistically significant F{\1, 622) = 5.045, p < .001, = .089, so 

further analyses were undertaken. Exhibit 31 demonstrates that teacher-reported quality of the 
service-learning project was correlated with multiple outcomes, including school attachment, 
civic knowledge, self-reported civic knowledge, civic skills, civic dispositions and civic 
engagement. These results confirm the results of prior research on the importance of service- 
learning quality, and help to explain the absence of significant overall effects for service-learning 
in this study on several outcome measures. 

Exhibit 31. Correiations Between Totai Teacher Ratings of Service-Learning 
Quaiity and Student Post-Test Outcomes 



Student Outcome 


N 


r 


P 


Academic Engagement 


667 


-.020 


.612 


Valuing School 


666 


-.077* 


.046 


School Attachment 


666 


.153*** 


.000 


Enjoyment of Math and Science 


646 


-.042 


.287 


Enjoyment of Reading/Eanguage Arts 
and Social Studies 


659 


-.007 


.848 


Civic Knowledge 


667 


.155*** 


.000 


Self-Reported Civic knowledge 


666 


.130** 


.001 


Civic Skills 


665 


.119** 


.002 


Community Attachment 


666 


.053 


.169 


Civic Dispositions 


665 


.101** 


.009 


Efficacy 


662 


.064 


.099 


Civic Engagement 


664 


.122** 


.002 



* p < .05, two-tailed test; ** p < .01 , two-tailed test, *** p < .001 , two-tailed test. 



The 38 items that measured quality in the teacher survey were derived in large part from the 
Essential Elements of Service-Eearning, a document generated by a coalition of practitioners in 
the field based on professional wisdom. The Essential Elements are: 
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1. Clear educational goals that require the application of concepts, content, and skills from the 
academic disciplines and involve students in the construction of their own knowledge. 

2. Student engagement in tasks that challenge and stretch them cognitively and 
developmentally. 

3. Assessment used as a way to enhance student learning as well as to document and evaluate 
how well students have met content and skill standards. 

4. Student participation in service tasks that have clear goals, meet genuine needs in the school 
or community and have significant consequences for themselves and others. 

5. Formative and summative evaluations employed in a systematic evaluation of the service 
effort and its outcome. 

6. Student voice in selecting, designing, implementing and evaluating the service project. 

7. Value of diversity as demonstrated by its participants, its practice and its outcomes. 

8. Direct communication and interaction with the community. 

9. Students prepared for all aspects of their service work including a clear understanding of 
task and role, the skills and information required by the task, awareness of safety 
precautions, and knowledge about and sensitivity to the people with whom they will work. 

10. Student reflection before, during, and after service, using multiple methods that encourage 
critical thinking, and serving as a central force in the design and fulfillment of curricular 
objectives. 

11. Multiple methods designed to acknowledge, celebrate, and validate students’ service work. 

To determine the relationships between specific Essential Elements and student outcomes, a 
correlation analysis was performed. Results, displayed in Exhibit 32, showed that the Elements 
varied greatly in their relationship to outcomes. Those Elements with the greatest positive 
correlations with outcomes were student engagement in challenging tasks; activities that meet 
genuine needs of the community; valuing diversity; direct contact with the community; and 
student preparation. Only 2 of the 1 1 Essential Elements, 3 and 9, were positively associated 
with several outcomes (specifically, academic engagement, valuing school, enjoyment of math 
and science, enjoyment of reading, language and social studies, and community attachment). 
Elements 5 and 1 1 were unrelated to any of the outcomes measured in this study, and several 
Elements had negative relationships to outcomes. 
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Exhibit 32. Correiations Between the Essentiai Eiements 
and Student Outcome Measures {N= 635) 













Service-Learning Element 










Student 

Outcome 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


Academic 

Engagement 


-.020 


.009 


.053 


.029 


-.034 


-.109* 


.125** 


-.054 


.142** 


-.069 


-.041 


Valuing School 


.057 


.050 


-.051 


-.008 


- 157** 


-.188** 


.176** 


-.109** 


.106** 


-.136** 


-.020 


School 

Attachment 


-.041 


.175** 


.130** 


.163** 


-.012 


.032 


-.026 


.167** 


-.011 


.142** 


-.040 


Enjoy Math/ 
Science 


-.032 


.001 


.043 


.017 


-.042 


-.130** 


.146** 


-.084* 


.153** 


-.103** 


-.021 


Enjoy Reading/ 
Language/Social 
Studies 


-.007 


.016 


.114** 


-.003 


-.033 


-.168** 


111** 


.057 


.150** 


-.045 


.200** 


Civic Knowledge 


.107** 


.202** 


-.033 


.125** 


-.037 


.124** 


-.067 


.193** 


- 112** 




-.011 


Self Reported 
Civic 

Knowledge 


.089* 


.066 


-.009 


.058 


.047 


.145** 


-.088* 


.153** 


-.038 


.162** 


-.047 


Civic Skills 


.074 


.162** 


.015 


.139** 


-.044 


.021 


.065 


.073 


.063 


O 

OO 

o^ 

* 


.005 


Community 

Attachment 


-.176** 


.005 


.175** 


.038 


.057 


.007 


- 118** 


.121** 


-.077 


.073 


-.049 


Civic 

Dispositions 




.181** 


-.078* 


.134** 


- Ill** 


.023 


.103** 


.062 


* 

00 

p 


.067 


-.006 


Efficacy 


.107** 


.123** 


-.070 


.090* 


-.089* 


.019 


.068 


-.013 


.026 


.039 


.064 


Civic 

Engagement 


.108** 


.154** 


.015 


.123** 


-.035 


.033 


.078* 


.098* 


.084* 


.089* 


-.027 



* p < .05; ** p < .01 , two-tailed test. 



These findings suggest the need to reconsider recommendations regarding the inclusion of all the 
Essential Elements in service-learning practice and to provide additional study to determine 
those Elements that are most critical in producing positive results for specific areas of desired 
outcomes. 

The service-learning student post-survey included several measures of students’ perceptions of 
aspects of their service-learning experience including the degree to which it had offered elements 
of quality, such as challenge, frequent reflection, and the sense of having made a contribution 
through service-learning. The relationships between these ratings of quality and outcomes were 
tested using multivariate analyses. 

Results showed a statistically significant overall difference between service-learning sites in 
students’ quality ratings F(64, 4,912) = 2.219, p < .001, Eta-squared = .028). Subsequent 
univariate tests revealed statistically significant differences in several specific quality elements 
including exposure to challenging tasks, making important decisions, discussing experiences 
with teachers, feeling criticized by adults, and feeling efficacious. Exhibit 33 displays the results 
of univariate tests for the Quality of the Service-Learning Experience scale, showing the areas of 
quality that had the greatest variation among schools. 
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Exhibit 33. Schooi Differences in Student Ratings 
of Quaiity of the Service-Learning Experience 











Partial Eta' 




df 


F 


P 


Squared 


Had real responsibilities. 


8 


1.120 


.348 


.014 


Had ehallenging tasks. 


8 


2.271 


.021 


.029* 


Made important deeisions. 


8 


2.366 


.016 


.030* 


Diseussed my experienee(s) 
with my teaeher(s). 


8 


3.264 


.001 


.041** 


Did things myself instead of 
observing. 


8 


1.611 


.118 


.021 


Had freedom to develop and 
use my own ideas. 


8 


1.695 


.096 


.022 


Adults eritieized me or my 
work. 


8 


2.930 


.003 


.037** 


Felt I made a eontribution. 


8 


3.195 


.001 


.040** 



* p < .05, two-tailed test; ** p < .01 , two-tailed test. 



A multiple regression analysis was conducted to determine whether there was a relationship 
between overall service-learning quality and students’ post- test scores. Results of this analysis 
were highly significant F(12, 600) = 28.041, p < .001, = .359. As seen in Exhibit 34, 

perceived quality of the service-learning experience was a strong, positive predictor of civic 
skills and efficacy. 

School Differences in Quality and Relationship to Student Outcomes 

The finding of positive effects for service-learning quality led to the hypothesis that quality 
differences between programs might have accounted for the lack of significant differences on 
some outcomes in the overall sample. Post hoc tests were conducted to examine program 
differences on elements of quality. Results showed significant between-program differences on 
the following dimensions of quality: 

• Perceived challenge of the service-learning tasks; 

• Students’ opportunities to make important decisions; 

• Discussion of service-learning experiences with teachers; 

• Perceived criticism by adults; and 

• Perceived efficacy. 

These findings may help explain why comparison students at several sites made greater or 
similar gains than service-learning students on the outcome measures. 
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Exhibit 34. Reiationships Between Service-Learning Quaiity 
and Student Outcomes 



Outcome Area 


Standardized Beta 
Coefficients 


t 


P 


Academic Engagement 


.063 


1.453 


.147 


Valuing School 


.025 


.538 


.591 


School Attachment 


.064 


1.392 


.164 


Enjoyment of Math and Science 


-.060 


-1.529 


.127 


Enjoyment of Reading/Eanguage 
Arts and Social Studies 


.030 


.754 


.451 


Civic Knowledge 


.003 


.101 


.920 


Self-Reported Civic Knowledge 


-.067 


-1.812 


.070 


Civic Skills 


.153 


3.508*** 


.000 


Community Attachment 


-.045 


-1.064 


.288 


Civic Dispositions 


.024 


.608 


.543 


Efficacy 


.423 


10.104*** 


.000 


Civic Engagement 


.075 


1.793 


.073 



p < .001, two-tailed test. 



Effects of Content Coverage 

Embedded within this study was the hypothesis that teaehers that explicitly addressed civic 
knowledge and skills and intentionally tried to influence civic dispositions would have more 
positive outcomes than those that were less explicit or intentional about their coverage. In the 
teacher survey, specific inquiries were made about content coverage of knowledge and skill areas 
measured in the student surveys, including concepts associated with government and political 
system, skills for participating within the formal political system, communities, and citizens’ 
responsibilities for active participation within their communities. 

Teachers in both groups covered various civics topics several times a week on average, as shown 
in Exhibit 35. However, teachers that implemented service-learning tended to cover civic 
content much less frequently than other teachers. Differences in frequency of coverage were 
most pronounced in the areas of how to debate an issue, citizens’ rights and responsibilities in a 
democracy, the branches of the government, respect for diversity, and current events. Given 
these large differences in content coverage, it is surprising that comparison students did not 
outscore their service-learning peers on most measures of civic knowledge and skills. Exhibit 36 
demonstrates that only enjoyment of reading and social studies and self-reported civic 
knowledge were significantly positively correlated with civics coverage scores. The results 
suggest that, by actively engaging students in community affairs and civic issues, service- 
learning may be as effective as content coverage in strengthening civic knowledge, skills, and/or 
dispositions. 
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Exhibit 35. Teachers’ Coverage of Civic-Reiated Subject Matter 





Service-Learning 




Comparison 




f Tests 






N 


Mean 


SD® 


N 


Mean 


SD 


df 


t 


P 


a. The U. S. Constitution 


14 


2.93 


1.685 


10 


3.00 


1.826 


22 


.098 


.922 


b. American history 


13 


3.15 


1.519 


11 


3.09 


1.758 


22 


.094 


.926 


c. World history 


14 


2.79 


1.626 


11 


3.09 


1.578 


23 


.472 


.641 


d. Current national events 


14 


4.07 


1.269 


11 


4.09 


1.375 


23 


.037 


.971 


e. Federal government 


14 


3.29 


1.637 


11 


3.55 


1.572 


23 


.401 


.692 


f. State government 


14 


2.43 


1.016 


11 


2.91 


1.375 


23 


1.006 


.325 


g. Local government 


14 


2.50 


.941 


11 


3.00 


1.265 


23 


1.135 


.268 


h. Rights of citizens in a democracy 


14 


3.36 


1.336 


11 


4.36 


.924 


23 


2.126 


.044 


i. Responsibilities of citizens in a 
democracy 


14 


3.36 


1.336 


11 


4.27 


.905 


23 


1.945 


.064 


j. Executive branch of the U.S. 
government 


14 


2.57 


1.399 


11 


3.36 


1.433 


23 


1.391 


.178 


k. Legislative branch of the U.S. 
government (Congress) 


14 


2.50 


1.286 


11 


3.18 


1.779 


23 


1.113 


111 


1. Cabinet roles 


13 


1.77 


.832 


11 


2.45 


1.572 


22 


1.365 


.186 


m. Judicial branch of the U.S. government 
(Supreme Court) 


14 


2.57 


1.342 


11 


3.27 


1.679 


23 


1.162 


.257 


n. Elections and the voting process 


14 


2.93 


1.328 


11 


3.36 


1.362 


23 


.804 


.430 


o. Analysis of political issues 

p. Controversial political issues and the 


14 


3.79 


1.369 


11 


3.82 


1.401 


23 


.058 


.954 


fact that there are multiple points of 


14 


3.93 


1.439 


11 


3.91 


1.221 


23 


.036 


.972 


view 




















q. Role of community organizations 


14 


3.00 


1.038 


11 


3.27 


1.421 


23 


.555 


.584 


r. Assessment of community needs 


14 


2.71 


.994 


11 


2.91 


1.446 


23 


.399 


.694 


s. How to debate an issue 


13 


2.31 


.855 


11 


3.45 


1.368 


22 


2.504 


.020 


t. How social change can come about 


13 


3.31 


.947 


10 


3.10 


1.663 


21 


.379 


.709 


u. Current state events 


14 


2.86 


1.167 


10 


3.70 


.823 


22 


1.957 


.063 


V. Current local events 


14 


2.86 


.864 


11 


3.82 


.982 


23 


2.600 


.016 


w. Volunteerism 


14 


2.86 


.949 


11 


2.91 


1.514 


23 


.105 


.917 


X. Respect for diversity 


14 


3.29 


1.267 


11 


4.18 


1.168 


23 


1.816 


.082 


y. Social problems (e.g., homelessness, 
drunk drivers) 


14 


3.29 


1.326 


11 


3.91 


1.044 


23 


1.277 


.214 


Total Summary Score 


14 


2.99 


.878 


11 


3.44 


.786 


23 


1.332 


.196 



Correlation coefficients were calculated between teacher responses to these questions. ^Standard deviation. 
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Exhibit 36. Correiations Between Civics Content Coverage 
and Student Civic Outcomes 



Student Outcome 


N 


r 


P 


Academic Engagement 


1013 


.014 


.666 


Valuing School 


1011 


-.020 


.521 


School Attachment 


1012 


.013 


.688 


Enjoyment of Math and Science 


979 


-.011 


.734 


Enjoyment of 

Reading/Eanguage Arts 
and Social Studies 


1004 


.126*** 


.000 


Civic Knowledge 


1013 


.028 


.375 


Self-Reported Civic Knowledge 


1012 


.093** 


.003 


Civic Skills 


1008 


-.016 


.612 


Community Attachment 


1012 


.034 


.282 


Civic Dispositions 


1008 


.006 


.843 


Efficacy 


1005 


-.034 


.284 


Civic Engagement 


1007 


.057 


.070 



** p < .01 , two-tailed test, *** p < .001 , two-tailed test. 



Teacher Characteristics as Moderators of Student Outcomes 

The study also tested the relationship between student outcomes and a variety of teacher 
characteristics and practices. A description of the pool of respondents to the teacher survey is 
presented in Exhibit 37. Most teachers in the sample taught multiple grade levels and nearly 
40% had been teaching for more than 20 years. The group was bimodal with regard to years 
teaching at the same school, with 43% at the school less than 5 years and 22% at the same school 
for over 21 years. Those that implemented service-learning had been doing so for an average of 
nearly 8 years. 

Years of Teaching and Student Outcomes 

One hypothesis tested within this study was that more experience in teaching would lead to 
stronger outcomes. Canonical correlation analysis indicated a highly significant relationship 
between teaching experience and student outcomes F( 12,949 = 4.050, p < .001, Eta-squared 
=.049. Using correlational analysis, the data showed that more years of teaching experience 
were significantly related to four student outcome measures: valuing school, r(l,l 10) = .103, p < 
.01; enjoying math and science, r(979) = .082, p < .05; civic skills, r(l,008) = .085, p < .01; and 
civic dispositions, r( 1,008) = .1 15, p < .001. In addition to teaching experience more generally, 
the study addressed whether teaching experience in service-learning was associated with stronger 
outcomes. Teachers’ experience with service-learning ranged from 2 to 24 years, with an 
average of about 8 years. Canonical correlation analysis revealed a highly significant 
relationship between teacher experience and student outcomes F(12,244 = 5.320), p < .001, Eta- 
squared = .113. Experience using service-learning was significantly positively associated with 
students’ post-test scores on three subscales: civic knowledge r(540) = .174,p <.001, civic 
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dispositions r(538) = .139, p < .01, and efficacy r(535) = .098,/? < .05. Years of using service- 
learning had a statistically significant negative association with community attachment r(539) = - 
.119,/?<.01. 

Exhibit 37. Description of Teacher Survey Respondents (Af = 24) 



Service-Learning Comparison 



Aggregate 





N 


Percent 


N 


Percent 


N 


Percent 


Teacher Respondents by Grade 


Grade 9 


6 


43 


4 


36 


10 


40 


Grade 10 


5 


36 


2 


18 


7 


28 


Grade 11 


9 


64 


8 


73 


17 


68 


Grade 12 


13 


93 


9 


82 


22 


85 


Years Teaching 


0 to 5 


1 


7 


3 


27 


4 


16 


6 to 10 


2 


14 


2 


18 


4 


16 


11 to 15 


4 


29 


0 


0 


4 


16 


16 to 20 


1 


7 


3 


27 


4 


16 


21 or more 


6 


43 


3 


27 


9 


36 


Years at the Same School 


0 to 5 


6 


43 


6 


55 


12 


98 


6 to 10 


2 


14 


1 


9 


3 


12 


11 to 15 


2 


14 


0 


0 


2 


8 


16 to 20 


0 


0 


2 


18 


2 


8 


2 1 or more 


4 


29 


2 


18 


6 


24 



Note: Teachers could choose more than one response. Percentages refer to proportion of teachers that answered 
the questions. 



Effects of the Frequency of Use of Active and Passive Instructionai Strategies 

Two additional hypotheses of this study were that service-learning teachers were more likely to 
use active teaching techniques than comparison teachers, and that service-learning would have 
an effect separate from and in addition to the effects of active learning. To test these hypotheses, 
teachers were asked to indicate how frequently they used various types of instructional strategies 
and materials, such as lecture, cooperative learning, supplemental reading materials, and so forth. 
Teaching techniques were classified as either active or passive. 



A MANOVA comparing service-learning teachers to comparison group teachers was not 
statistically significant F{A, 16), p = .426, Eta-Squared = .204 for the four passive instruction 
techniques, even though the effect size was moderate. As shown in Exhibit 38, which displays 
descriptive statistics, service-learning teachers tended to use passive and active instructional 
techniques at about an equal rate while comparison teachers more frequently employed passive 
techniques although these differences are not statistically significant. A MANOVA comparing 
service-learning teachers to comparison teachers regarding use of active techniques was not 
significant F(1 , 15) =.753, p = .633, Eta-squared = .260 although the effect size was moderate. 
Among the passive techniques, the largest differences in frequency of use were for textbook 
reading and use of video, with both implemented more often by comparison teachers. Among 
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the active techniques, the greatest differences were found in the use of cooperative learning, 
visits to government or community institutions, debates or discussion, and research reports. Of 
all of the techniques listed, service-learning teachers used cooperative learning, debates or 
discussions, and lectures most often while comparison teachers implemented debates or 
discussions, lectures, textbook reading, and cooperative learning most often, although differences 
were not statistically significant. 

Canonical correlations between the use of active versus passive instruction and student 
outcomes were determined for all teachers. Exhibit 39 shows bivariate correlations. Active 
learning strategies were statistically significantly associated with multiple student outcomes, 
including very strong relationships with valuing school, enjoying reading/language arts/social 
studies, self-reported civic knowledge, civic skills, and civic dispositions. Academic 
engagement and efficacy were also impacted by active teaching. Passive strategies had 
statistically significant negative relationships with civic engagement and civic dispositions. 
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Exhibit 38. Types of instructionai Strategies Used in the Ciassroom 



Service-Learning Comparison t Tests 





N 


Mean 


SD 


N 


Mean 


SD 


df 


t 




Passive 


Lectures 


14 


3.50 


1.345 


10 


3.80 


1.229 


22 


.558 


.583 


Textbook reading 


13 


2.69 


1.182 


10 


3.60 


1.265 


21 


1.771 


.091 


Videos, DVDs, or television 


13 


2.38 


.650 


10 


2.70 


1.337 


21 


.747 


.464 


Multiple choice tests 


13 


2.46 


1.050 


10 


2.40 


1.075 


21 


.138 


.892 


Passive Subscale 


14 


2.79 


.774 


10 


3.13 


.648 


22 


1.130 


.271 


Active 


Visits to government or 


14 


1.64 


.633 


10 


1.20 


.422 


22 


1.922 


.068 


community institutions 


Debates or discussions 


14 


3.64 


1.336 


10 


3.90 


1.197 


22 


.485 


.633 


Mock trials, role plays, or 


14 


2.43 


1.089 


10 


2.50 


1.509 


22 


.135 


.894 


other simulations 


Assignments in which 


14 


2.64 


1.082 


10 


2.70 


1.636 


22 


.103 


.919 


students analyze media 
presentations of 
information 


Research reports 


14 


2.42 


.852 


10 


2.20 


1.135 


22 


.565 


.578 


Student generated projects 


14 


2.86 


1.099 


10 


2.70 


1.337 


22 


.316 


.755 


Cooperative learning 


13 


4.08 


1.188 


10 


3.60 


1.265 


21 


.928 


.564 


Active Subscale 


14 


2.79 


.646 


10 


2.69 


.824 


22 


.350 


.729 



Note: Subscales are the averages of the items listed in each subsection. Two-tailed f tests were conducted between service-learning teacher responses and 
comparison group teacher responses to items and subscales. None of the results were statistically significant. 
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Exhibit 40. Correiations Between Active and Passive Teacher Pedagogy and Student Post-Test Outcomes 



Teacher Pedagogy 

Total Number in 
Sample 






Post-Test Student Perceptions 




Academic 

Engagement 


Valuing 

School 


School 

Attachment 


Enjoyment of 
Math and 
Science 


Enjoyment of 
Reading/Language 
Arts and Social 
Studies 


Civic 

Knowledge 


Active Instruction 


.079* 


.135*** 




.102** 




.093** 


N 


(910) 


(910) 


(909) 


(882) 


(902) 


(910) 


Passive Instruction 


.051 


-.003 


-.057 


.047 


-.099** 


1.010** 


N 


(910) 


(908) 


(909) 


(882) 


(902) 


(910) 



* p < .05, two-tailed test; ** p < .01 , two-tailed test; *** p < .001 , two-tailed test. A/ of cases in parentheses below each correlation. 



Post-Test Student Perceptions 

Teacher 

Pedagogy 



Total Number in 
Sample 


Self-Reported 
Civic Knowledge 


Civic Skills 


Community 

Attachment 


Civic 

Dispositions 


Efficacy 


Civic 

Engagement 


Active Instruction 


-.031 




.057 


.151*** 


.080* 


.166*** 


N 


(909) 


(905) 


(909) 


(905) 


(903) 


(905) 


Passive Instruction 


-.146*** 


-.052 


.020 


-.070* 


-.025 


-.072* 


N 


(909) 


(905) 


(909) 


(905) 


(903) 


(905) 



* p < .05, two-tailed test; ** p < .01 , two-tailed test; *** p < .001 , two-tailed test. A/ of cases in parentheses below each correlation. 
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To determine whether the aetive or passive nature of instruetion explained the effeets of serviee- 
learning (rather than the pedagogy of service-learning itself), a MANCOVA was conducted with 
teacher active and passive scores used as covariates and student group used as a categorical 
variable. Results are presented in Exhibit 40. The use of active strategies explained about 9% of 
the variance in overall student outcomes, and passive instruction accounted for approximately 
7%. Participation in service-learning explained an additional 3% of the variance in student 
outcomes when type of pedagogy was statistically controlled. This indicates a small positive 
influence for service-learning beyond active pedagogy alone. 

Exhibit 40. Infiuence of Type of Instruction (Active vs. Passive) 

on Student Outcomes 



Effect 


df 


F 


P 


Partial 

Eta- 

squared 


Intercept 


12 


80.413 


.000*** 


.531 


Active 


12 


6.847 


.000*** 


.088 


Instruction 


Passive 


12 


4.903 


.000*** 


.065 


Instruction 


Student group 


12 


2.025 


.020* 


.028 



* p< .05, two-tailed test; *** p < .001 , two-tailed test. 



Univariate analyses of the effects of active vs. passive teaching and of service-learning 
participation controlling for type of pedagogy revealed that active teaching explained significant 
amounts of the variance on all outcome measures, except civic knowledge. When active and 
passive instruction were controlled, service-learning students scored significantly higher on the 
valuing school measure (M = 3.612) than comparison students (M = 3.496), while, in contrast 
comparison students reported greater enjoyment of reading and social studies {M = 3.611) versus 
service-learning reading and social studies (M = 3.499). Exhibit 41 displays univariate results. 
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Exhibit 41. Effects of Service-Learning Independent of Type of Instruction 
(Active V. Passive) Generaiiy Used by the Teacher 



Group 

(Service-Learning 

Active Instruction Passive Instruction vs. Comparison) 



Dependent Variable 


P 


Partial 

Eta- 

squared 


P 


Partial 

Eta- 

squared 


p 


Partial 

Eta- 

squared 


Academic Engagement 


.002 


.011** 


.023 


.006* 


.623 


.000 


Valuing School 


.000 


.022*** 


.104 


.003 


.040 


.005* 


School Attachment 


.000 


.031*** 


.634 


.000 


.250 


.002 


Enjoyment of Math 


.000 


.016*** 


.008 


.008** 


.565 


.000 


and Science 














Enjoyment of 


.008 


.008** 


.014 


.007* 


.027 


.006* 


Reading/Eanguage Arts 














and Social Studies 














Civic Knowledge 


.065 


.004 


.041 


.005* 


.662 


.000 


Self-Reported 


.011 


.008* 


.000 


.028*** 


.937 


.000 


Civic Knowledge 














Civic Skills 


.000 




.880 


.000 


.177 


.002 


Community Attachment 


.021 


.006* 


.092 


.003 


.093 


.003 


Civic Dispositions 


.000 


.020*** 


.595 


.000 


.232 


.002 


Efficacy 


.023 


.006* 


.860 


.000 


.875 


.000 


Civic Engagement 


.000 


.021*** 


.502 


.001 


.876 


.000 



* p < .05, two-tailed test; ** p < .01 , two-tailed test, *** p < .001 , two-tailed test. 
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Conclusions 



This study examined the impact of participation in service-learning on high school students’ 
attitudes toward school and civic development. Using a national sample of classrooms of 
students that participated in service-learning matched with classrooms of students of similar 
demographic and achievement background that did not participate in service-learning, the study 
investigated the effect of service-learning participation on students’ academic engagement, 
valuing school, attachment to school, and civic knowledge, skills, dispositions, and activities. 
Additionally, the study examined the degree to which many of the variables related to the 
service-learning experience served as moderators of outcomes. 

A variety of methodologies was used to collect data. Students in service-learning classes and 
students at the same or a demographically matched school at the same grade level taking classes 
in the same content area responded to surveys in the fall and spring. Teachers completed surveys 
in the spring. Students and teachers also participated in focus groups, classrooms were observed, 
and administrators were interviewed. Subscales were created from survey items and differences 
between students that participated in service-learning relative to those that did not participate 
were analyzed, both in terms of the overall sample and using the matched classroom design. 

Results showed: 

• Although service-learning students scored higher than comparison students on several 
outcomes, most differences were not statistically significant. 

• Service-learning students had significantly higher scores on enjoyment of school overall 
than comparison group peers. No differences were found in enjoyment of specific 
content areas. 

• Service-learning students were significantly more likely than comparison group 
members to report that they intended to vote. 

The students in this study that participated in service-learning had very different experiences 
which led to wide variations in student perceptions of service-learning and the impact that it had. 
Analysis of matched pairs of students showed that just a few of the classrooms in the sample 
accounted for many of the differences. 

• A variety of hypotheses related to the service-learning experiences and outcomes were tested. 
The hypothesis that service-learning was inherently engaging was supported in some 
schools and in some ways. Over half of the participants in service-learning said that they 
often or always work hard on their service-learning projects, pay attention, and try to learn 
from them. 

• Engagement in service-learning was a strong predictor of all student outcomes, except 
for the civic knowledge measure. Students who reported stronger engagement in service- 
learning were statistically significantly more likely to be academically engaged, value 
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schooling, become attaehed to sehool and eommunity, enjoyed eontent eourses, pereeive a 
gain in civic knowledge, skills, and dispositions, become more civically engaged in general, 
and felt greater efficacy. 

• Students that chose the issue for their service-learning project made greater gains on 
the objective questions of civic knowledge than others, while those that worked on a 
teacher-selected civic issue made greater gains than those that worked on an 
environmental service project. Students who worked on the teacher seleeted eivic 
activities perceived that they made greater gains in civic knowledge than others. 

• The type of service activity (direct, indirect, or civic or political action) performed was 
related to nearly all outcomes measures. Participating in civic or political action was 
positively related to civic knowledge and civic dispositions. Direct service activities were 
associated with community attachment, while indirect activities were assoeiated with higher 
post-test scores on academic engagement, valuing school, and enjoyment of specifie subject 
matters. 

• Duration of the service-learning experience was also significantly related to most of the 
outcome areas. Generally, those service-learning activities that were one to two months in 
duration had the highest academic impacts while those that were a semester long had the 
greatest civic impacts. 

• All of the civic outcome areas except community attachment were strongly related to 
teacher-reported service-learning program quality. School attachment was also strongly 
related to program quality. However, the most commonly used indicators of quality: the 
Essential Elements of Service-Learning, did not predict outcomes evenly. Rather, specific 
Elements were much more highly related to outcomes and some Elements were not related to 
outcomes at all. 

• Teachers that implemented service-learning were less likely than other teachers to 
explicitly cover content areas related to civics. However, civics content coverage was 
related to only two student outcomes: self-reported civic knowledge and enjoyment of 
reading/language arts and social studies. 

• Teacher characteristics were also found to be related to outcomes. Number of years of 
teaching experience was significantly related to students’ valuing school, enjoying math 
and science, civic skills, and civic dispositions. Longer experience using service-learning 
was associated with higher civie knowledge, civic dispositions, and efficacy scores. 

• Service-learning teachers were more likely to use a variety of active strategies more 
often than teachers that did not implement service-learning. The use of active strategies, 
in turn, was highly related to student outcomes in the areas of valuing sehool, attachment to 
school, enjoyment of content areas, aequisition of civic knowledge, skills, and dispositions, 
and civic engagement. 



The Impact of Participation in Service-Learning on 
High School Students’ Civic Engagement 



54 



RMC Research Corporation, Denver, CO 



• The use of active instructional strategies in general explained about 9% of the variation 
in overall outcomes. The use of service-learning accounted for an additional 3% of the 
positive outcomes. 

These data show that students had vastly different experiences with service-learning during the 
year, and these differences illuminate potential reasons for the results. As repeatedly shown in 
service-learning research, there is a strong need for high quality service-learning practice, 
whether quality is measured directly through quality indicators, or less directly, through 
assessments of student engagement in service-learning. 

Limitations of the Study 

As is often the case, the results of this study call for more investigation in order to yield greater 
understanding. The study has multiple limitations. First, the study relies heavily on self-report. 
While it has a strong matched comparison group design, the design was undermined by the fact 
that there was very uneven quality within the sites so that the main hypotheses could not be 
tested well. There were too few controls on the parameters and content of service-learning to 
provide definitive conclusions, though the data were suggestive of many of the variables that 
served as moderators of outcomes. No longitudinal data have yet been collected. 

The qualitative data acquired as part of the study may serve to illuminate more of the differences 
between schools in the quality and content of service-learning and will be used to conduct finer 
grained analyses in the second year of the project. In addition, more data will be collected on 
various aspects of quality, particularly those found to moderate outcomes, so that greater 
understanding of the ways in which they serve to influence impact can be understood. For 
example, next year, teachers will be asked to document specific teaching practices and ways in 
which they connect service-learning to those practices. The degree to which service-learning 
plays an additive role beyond the use of active instructional strategies will be further 
investigated. The study will also look at the role of student characteristics in order to determine 
which students benefit most under what conditions. 

The results of this study can best be described as suggestive and worthy of consideration by 
service-learning practitioners in the field. When service-learning is of high quality, at best it 
appears to be as good as other instructional strategies in producing civic engagement outcomes. 
However, more data are needed to understand how impact can be maximized. 
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CIRCLE (The Center for Information and Research on Civic Learning and Engagement) promotes research 
on the civic and political engagement of Americans between the ages of 15 and 25. Although CIRCLE 
conducts and funds research, not practice, the projects that we support have practical implications 
for those who work to increase young people's engagement in politics and civic life. CIRCLE is also a 
clearinghouse for relevant information and scholarship. CIRCLE was founded in 2001 with a generous 
grant from The Pew Charitable Trusts and is now also funded by Carnegie Corporation of New York. It is 
based in the University of Maryland's School of Public Policy. 
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